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To our parents 




Preface 



This book is about three seemingly independent areas of mathematics: 
combinatorial group theory, the theory of Lie algebras and affine algebraic 
geometry. Indeed, for many years these areas were being developed fairly 
independently. Combinatorial group theory, the oldest of the three, was 
born in the beginning of the 20th century as a branch of low-dimensional 
topology. Very soon, it became an important area of mathematics with its 
own powerful techniques. 

In the 1950s, combinatorial group theory started to influence, rather 
substantially, the theory of Lie algebras; thus combinatorial theory of Lie 
algebras was shaped, although the origins of the theory can be traced back 
to the 1930s. 

In the 1960s, B. Buchberger introduced what is now known as Grobner 
bases. This marked the beginning of a new, “combinatorial” , era in commu- 
tative algebra. It is not very likely that Buchberger was directly influenced 
by ideas from combinatorial group theory, but his famous algorithm bears 
resemblance to Nielsen’s method, although in a more sophisticated form. 
More recently, in the 1990s, ideas from combinatorial group theory started 
to have a more direct and significant impact on the areas of commutative 
algebra and affine algebraic geometry through the work of several mathe- 
maticians including the authors of this book. It is remarkable that, also in 
the 1990s, ideas from algebraic geometry found their way into combinato- 
rial group theory through the work of G. Baumslag, A. G. Myasnikov, 
V. Remeslennikov, O. Kharlampovich and B. I. Plotkin, thus creating 
a “two-way traffic” of ideas that allows algebraic geometry to shed light on 
group theory and vice versa. 
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The main purpose of this book is to show how ideas from combinatorial 
group theory have spread to the other two areas, with the main focus on 
the area of commutative algebra where the influence of these ideas has been 
especially spectacular. 

We would like to emphasize that we only consider here purely combinato- 
rial methods and results; in particular, we leave out important interactions 
of group theory with topology and geometry that were recently used with 
great success in solving several difficult problems about free groups and 
their automorphisms. We also leave out geometric methods in affine al- 
gebraic geometry and concentrate on combinatorial ones, in particular on 
those that come from combinatorial group theory. 

The book is targeted at research mathematicians as well as graduate 
students with an interest in the general area of algebra. 
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Introduction 



The main purpose of this book is to show how ideas from combinatorial 
group theory have spread to two other areas of mathematics: the the- 
ory of Lie algebras and affine algebraic geometry. Some of these ideas, 
in turn, came to combinatorial group theory from low-dimensional topol- 
ogy in the beginning of the 20th century through the work of several 
prominent topologists, such as M. Dehn, H. Tietze, J. Nielsen, H. Reide- 
meister, J. H. C. Whitehead, to name just a few. Very soon, combinatorial 
group theory became an independent area of mathematics with its own 
powerful techniques. New, purely combinatorial methods were created by 
O. Schreier, R. Fox, W. Magnus, A. G. Kurosh, R. C. Lyndon, and many 
others. 

Then, in the 1950s, mostly through the work of A. I. Shirshov, sev- 
eral ideas from combinatorial group theory (most notably, Nielsen’s and 
Schreier’s techniques) have been adopted in the theory of Lie algebras; 
thus the combinatorial theory of Lie algebras was shaped, although the 
origins of the theory can be found in the papers (published in the 1930s) 
by P. Hall, W. Magnus, and E. Witt on the structure of free and nilpotent 
groups. 

In the 1960s, combinatorial methods came to commutative algebra when 
B. Buchberger introduced what is now known as Grobner bases. It is not 
very likely that Buchberger was directly influenced by ideas from combi- 
natorial group theory, but his famous algorithm bears a resemblance to 
Nielsen’s method, although in a more sophisticated form. 

This book is in three fairly independent parts. In Part I, we first give 
a very brief exposition of several classical techniques in combinatorial group 
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theory, namely methods of Nielsen, Whitehead, and Tietze. We do not go 
into details because there are two very well-established monographs where 
a complete exposition of these techniques is given. For an exposition of 
Nielsen’s method, we recommend [231], whereas [227] has, in our opinion, 
a better exposition of Whitehead’s and Tietze’s methods. Then we go on to 
very recent ideas and results. We focus here on the theory of free groups and 
their automorphisms since we believe that it is in the heart of combinatorial 
group theory. This does not mean, of course, that other areas, for instance 
the theory of braid groups or hyperbolic groups, are less significant. 

We would also like to say up front that we only consider here purely 
combinatorial ideas and results; in particular, we leave out important in- 
teractions of group theory with topology and geometry. It should be pointed 
out that topological methods were recently used with great success in solv- 
ing several difficult problems about free groups and their automorphisms. 
We only mention here seminal papers of J. R. Stallings [373] and M. Bestv- 
ina and M. Handel [51], but, in fact, many other people, notably G. Levitt, 
M. Lustig, M. Feighn, Z. Sela, E. Turner, J. McCool, W. Dicks, E. Ven- 
tura, K. Vogtmann, and S. Krstic, have contributed greatly to this area by 
employing topological techniques. 

Part II contains the main focus of this book. Here, we show how the 
aforementioned techniques of combinatorial group theory have found their 
way into affine algebraic geometry, a fascinating area of mathematics that 
studies polynomials and polynomial mappings. An interesting thing about 
this area is that most of the research here is focused on five or six outstand- 
ing problems. The statements of those problems are rather elementary and 
can be understood by an average high school student. However, the prob- 
lems have so far eluded all attempts at a complete solution, even though 
some of them have been settled in low dimensions. The relevant bibliogra- 
phy is really vast; however, most people who work(ed) in this area use(d) 
geometric methods. Notable exceptions are H. Bass, P. M. Cohn, W. Dicks, 
A. van den Essen, L. Makar-Limanov, J. McKay, S. S. Wang, and D. Wright. 

The first major result about polynomial automorphisms that bears re- 
semblance to the corresponding result in combinatorial group theory was 
the Jung-van der Kulk theorem [194] describing generators of the group of 
automorphisms of a polynomial algebra in two variables over a field. Then, 
more than 40 years later, the parallelism between the two areas came into 
play again, starting with the paper [124] and gradually attracting the at- 
tention of other people, including the authors of this book, V. Drensky, 
A. van den Essen, Z. Jelonek, W. Neumann, P. Wightwick, and others. 
One of the most significant results so far obtained by employing ideas from 
combinatorial group theory (in this case, Whitehead’s idea of “peak reduc- 
tion” ) is a recent solution of the automorphic equivalence problem for plane 
curves by Wightwick [401]. We believe that other ideas from combinatorial 
group theory also have a great potential in affine algebraic geometry, which 
was one of the main reasons for us to write this book. 
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It is remarkable that around the same time, in the mid-1990s, ideas 
from algebraic geometry found their way into combinatorial group theory. 
Perhaps an earlier paper of Lyndon [226] had a glimpse of such interac- 
tion, but it became really convincing through the work of G. Baumslag, 
A. G. Myasnikov, V. Remeslennikov, O. Kharlampovich [41, 176, 177, 285], 
and, essentially independently, B. I. Plotkin [308]. Thus, we have now a 
“two-way traffic” of ideas that allows algebraic geometry to shed light on 
group theory and vice versa! 

In Part III, we show how ideas from combinatorial group theory con- 
tributed to the theory of free algebras. The focus here is on Schreier 
varieties of algebras (a variety of algebras is said to be Schreier if any 
subalgebra of a free algebra of this variety is free in the same variety of 
algebras). This includes varieties of Lie (super)algebras, of all algebras, and 
some others. Free algebras in these varieties admit an adoption of Nielsen’s 
and Schreier’s methods, which leads to a substantial parallelism between 
ideas and results in the two areas. We start Part III with basic proper- 
ties of these algebras, Schreier’s technique, free differential calculus, and 
differential separability of subalgebras. Then we consider automorphic or- 
bits, combinatorial properties of (generalized) primitive elements, and fixed 
points of endomorphisms. Then we consider the variety of Leibniz algebras. 
Although it is not a Schreier variety, many of its combinatorial properties 
are similar to those of Schreier varieties. 

We conclude the Introduction by comparing the present book to previ- 
ously published ones on similar subject (s). There is an excellent monograph 
by A. van den Essen [122] that has some overlap with Part II of our book. 
We were trying to avoid too much overlap by focusing our attention here 
specifically on interactions between combinatorial group theory and affine 
algebraic geometry and not trying to survey all recent work in the latter 
area. We just give a very brief survey, in the beginning of each chapter, 
of the work related to a particular problem. Then, our Part III has some 
overlap with the monograph [275]; however, since [275] was published, there 
has been a lot of new work done in combinatorial theory of free algebras, 
so we emphasize this new work here. 




Part I 



Groups 




Introduction to Part I 



In this part, we first give a brief exposition of several classical techniques in 
combinatorial group theory, namely methods of Nielsen, Schreier, White- 
head, and Tietze (Chapter 1). We will need this in the subsequent parts of 
the book to show how these basic ideas of combinatorial group theory have 
penetrated affine algebraic geometry and the theory of Lie algebras. Also 
in Chapter 1, we give a brief exposition of Fox differential calculus; this 
technique, although not very popular among group theorists these days, 
sometimes provides new insights and suggests new directions of research. 
Also, it plays a rather important role in combinatorial theory of Lie al- 
gebras, where we do not have the facility of using powerful topological or 
geometrical methods. 

Then we go on to recent ideas and results in the theory of free groups and 
their automorphisms. We leave out here important interactions of group 
theory with topology and geometry and focus on purely combinatorial 
methods having in mind, again, their future use in other parts of the book. 

Chapter 2 is about test elements. The idea here is to distinguish, for ex- 
ample, automorphisms among arbitrary endomorphisms by means of their 
action on a single element, a test element. The first result of this particular 
type was that of Nielsen [293], who proved that if an endomorphism of the 
free group of rank 2 fixes the commutator of a pair of generators, then it 
is an automorphism. Similar results for other particular elements of a free 
group were later obtained by Zieschang [417, 418], Rosenberger [318], Dold 
[100], Rips [314], Durnev [115], and others. Shpilrain [352, 355] generalized 
most of these results and put them in perspective. Turner [383] character- 
ized test elements of a free group as elements that do not belong to any 
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proper retract. Based on this characterization, Comerford [82] suggested an 
algorithm for recognizing test elements of a free group. The idea behind the 
definition of a test element has numerous ramifications; for example, one 
can consider properties (other than being an automorphism) of an endo- 
morphism that can be recovered from its action on a single element. Also, 
one can consider test sets rather than just test elements. We discuss these 
and other ramifications in Chapter 2. 

In Chapter 3, we consider various generalizations of primitive elements: 
generic elements in the sense of Stallings [375], almost primitive elements 
in the sense of Rosenberger [318], and A- primitive elements in the sense of 
Shpilrain [356]. It turns out that all these generalizations are related, but re- 
lations between them are rather nontrivial, which opens several interesting 
research avenues. 

In the concluding chapter, Chapter 4, we deal with automorphic orbits in 
a free group F n of finite rank; these are sets of the form {v G F n , v = <p(u) 
for some <p G Aut(F n ) and a fixed u G F n }. One special and very in- 
teresting automorphic orbit is the set of all primitive elements (these are 
automorphic images of a free generator of F n ). In particular, the follow- 
ing problem, along with its generalizations [352], has been a source of 
inspiration for several people: if an endomorphism (p of a free group F n 
takes every primitive element to another primitive, is (p an automorphism? 
Recently, D. Lee [205] has settled this problem in the affirmative for ev- 
ery n. Other interesting problems arise from considering “abridged” orbits 
{v G F n ; \v\ = |m|, (p(v) = u for some ip G Aut(F n )}. For instance, the 
problem of counting primitive elements of a given length (see Problem (FI 7) 
in [42]) has received considerable attention lately. 




1 

Classical Techniques 
of Combinatorial Group Theory 



In this chapter, we give a brief exposition of several classical techniques in 
combinatorial group theory, namely methods of Nielsen, Schreier, White- 
head, and Tietze and the free differential calculus due to Fox. We do not go 
into details here because there are two very well-established monographs 
where a complete exposition of these techniques is given. For an exposition 
of Nielsen’s and Schreier ’s methods, we recommend [231], whereas [227] has, 
in our opinion, a better exposition of Whitehead’s and Tietze’s methods. 

We emphasize the aspects of classical techniques of combinatorial group 
theory here that can be carried on to other areas of mathematics considered 
in this book, namely to affine algebraic geometry and the theory of free 
algebras. 



1.1 Nielsen’s Method 

Let F = F n be the free group of a finite rank n > 2 with a set 
X = {#i, . . . , x n } of free generators. Let Y = {?/i, . . . , y m } be an arbitrary 
finite set of elements of the group F. Consider the following elementary 
transformations that can be applied to Y: 

(Nl) yi is replaced by yiPj or by yjpi for some j ^ i; 

(N2) yi is replaced by y~ x \ 

(N3) pi is replaced by some yj, and at the same time pj is replaced by yi\ 
(N4) delete some pi if pi — 1. 
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It is understood that yj does not change if j 7^ i. 

Every finite set of reduced words of F n can be carried by a finite sequence 
of Nielsen transformations to a Nielsen-reduced set U = {ui, . . . i.e., 

a set such that for any triple v\, V2, v$ of the form uf 1 , the following three 
conditions hold: 

(i) vi ^ i; 

(ii) V1V2 ± 1 implies |^i^ 2 | > |t>i|, \v2 1; 

(iii) viv 2 ± 1 and V2V3 ^ 1 implies \v\V2V3\ > |vx| - 1^1 + |v3|. 

It is easy to see that if U = (wi, U2, . . . Uk) is Nielsen-reduced, then the 
subgroup of F n generated by U is free with a basis U. 

One might notice that some of the transformations (N 1 )-(N 4 ) are redun- 
dant; i.e., they are compositions of other ones. The reason behind that we 
will explain below. 

We say that two sets Y and Y are Nielsen equivalent if one of them 
can be obtained from another by applying a sequence of transformations 
(N 1 )-(N 3 ). It was proved by Nielsen that two sets Y and Y generate the 
same subgroup of the group F if and only if they are Nielsen equivalent. 
This result is now one of the central points in combinatorial group theory. 

Note, however, that this result alone does not give an algorithm for decid- 
ing whether or not Y and Y generate the same subgroup of F. To obtain an 
algorithm, we need to somehow define the complexity of a given set of ele- 
ments and then show that a sequence of Nielsen transformations (N 1 )-(N 3 ) 
can be arranged so that this complexity decreases (or, at least, does not 
increase) at every step (this is where we may need “redundant” elementary 
transformations! ) . 

This was also done by Nielsen; the complexity of a given set Y = 
{yi , • • • > 2/m} is just the sum of the lengths of the words y\ , . . . , y m . Now the 
algorithm is as follows. Reduce both sets Y and Y to Nielsen-reduced sets. 
This procedure is finite because the sum of the lengths of the words de- 
creases at every step. Then solve the membership problem for every element 
of Y in the subgroup generated by Y and vice versa. 

Nielsen’s method therefore yields (in particular) an algorithm for decid- 
ing whether or not a given endomorphism of a free group of finite rank is 
an automorphism. 

Another classical method that we sketch in this section is that of Schreier. 
We are going to give a brief exposition of this method here in a special case 
of a free group; however, it is valid for arbitrary groups as well. We are 
going to use the same notation that we used before. 

Let if be a subgroup of F. A right coset representative function for F (on 
the generators Xi) modulo if is a mapping of words in x*, w(x 1, #2, • • • ) — ^ 
w(x i,X2,...)> where the w(x i,X2,...) form a right coset representative 
system for F modulo if, which contains the empty word, and where 
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w(x 1 ,^ 2 , . . . ) is the representative of the coset of w(x i, X 2 , . . . ). Then we 
have: 

Theorem 1.1.1. Ifw^wisa right coset function for F modulo H , then 
H is generated by the words 

UXi • UX~i ~ l , 

where u is an arbitrary representative and Xi is a generator of F. 

This already implies, for instance, that if F is finitely generated and H 
is a subgroup of finite index, then H is finitely generated. 

Furthermore, a Schreier right coset function is one for which any initial 
segment of a representative is again a representative. The system of repre- 
sentatives is then called a Schreier system. It can be shown that there is 
always some Schreier system of representatives for F modulo H. Also, there 
is a minimal Schreier system ; i.e., a Schreier system in which each repre- 
sentative has a length not exceeding the length of any word it represents. 
Not every Schreier system is minimal. 

Example 1.1.2. Let F be the free group on a and &, and let H be the nor- 
mal subgroup of F generated by a 2 , 6 2 , and a6a _1 6 _1 . Then F/H has four 
cosets. The representative system {1, a, 6, ab} is Schreier, as is the represen- 
tative system {l,a, b, ab~ 1 }. The representative system {1, a, 6, a~ 1 b~ 1 } is 
not Schreier; for, the initial segment a -1 of a~ 1 b~ 1 is not a representative. 

Using a Reidemeister-Schreier rewriting process , one can obtain a pre- 
sentation (by generators and relators) for H. This implies, among other 
things, the following important result which has inspired a lot of research 
in other algebraic systems; some of this research is described in Part III of 
this book. 

Theorem 1.1.3 (Nielsen— Schreier). Every nontrivial subgroup H of 
a free group F is free. 

One can effectively obtain a set of free generating elements for H\ namely, 
those UXi ' uxi~ l such that UXi is not freely equal to a representative. 
Schreier obtained this set of free generators for H in 1927. In 1921, Nielsen, 
using quite a different method, had constructed a set of free generators 
for H if F was finitely generated. The generators of Nielsen are precisely 
those Schreier generators obtained when a minimal Schreier system is used. 
We refer to [231, Chapter 3] for more details. 



1.2 Whitehead’s Method 

A more difficult problem is, given a pair of elements of a free group F, to find 
out if one of them can be taken to the other by an automorphism of F. We 
call this problem the automorphic conjugacy problem. It was addressed by 
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Whitehead, who came up with another kind of elementary transformation 
in a free group. Let X = {xi, X 2 , . . . } be a (finite) set of generators of the 
group F, and let X~ l = {x± 1 , X 2 \ . . . } 

(Wl) permutations on the set X ±l = X U X~ l . 

(W2) a transformation u of the second type is defined by a set S C X ±x 
and a letter a £ X ±x with a £ 5 and a~ l £ S in such a way that if 
x £ X ±x , then 

(a) uj(x) = x provided x = a ±1 ; 

(b) u(x) = xa provided x ^ a, x £ S and x~ x £ 5; 

(c) u)(x) = a~ x xa provided both x,x _1 £ 5; 

(d) u{x) = x provided both x,x~ l £ S. 

One might notice some similarity between Nielsen and Whitehead 
transformations. However, they differ in one essential detail: Nielsen trans- 
formations are applied to arbitrary sets of elements, whereas Whitehead 
transformations are applied to a fixed basis of the group F. 

Using (informally) matrix language, we can say that Nielsen transfor- 
mations correspond to elementary row transformations of a matrix (this 
correspondence can actually be made quite formal — see [353]), whereas 
Whitehead transformations correspond to conjugations (via changing the 
basis). This latter type of matrix transformation is known to be more 
complex, and the corresponding structural results are deeper. 

There is pretty much the same relation between Nielsen and Whitehead 
transformations in a free group. 

Note also that the Whitehead transformation (W2) is rather complex 
and it does not really look so “elementary”. This is, again, in order to 
be able to arrange a sequence of elementary transformations so that the 
complexity of a given element (in this case, just the lexicographic length 
of a cyclically reduced word) would decrease (or, at least, not increase) 
at every step. To prove that this is possible, Whitehead has introduced a 
remarkable method called “peak reduction”. This means that if in a se- 
quence of elementary transformations the complexity goes up (or remains 
unchanged) before eventually going down, then there must be a pair of sub- 
sequent elementary transformations in the sequence (a “peak”) such that 
one of them increases the complexity (or leaves it unchanged) and then 
the other one decreases it. Then one tries to prove that such a peak can 
always be reduced; i.e., replaced by (perhaps a sequense of) elementary 
transformations such that the complexity decreases at every step in the 
sequense. 

This arrangement still leaves us with a difficult problem — to find out if 
one of two elements of the same complexity (meaning of the same length) 
can be taken to the other by an automorphism of F. This was always con- 
sidered the most time-consuming part of Whitehead’s algorithm, although 
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recent results in [286] make it appear likely that this part of the algorithm 
is fast, too. 

In one special case, the second part of the algorithm is just not needed, 
namely where one of the elements is primitive ; i.e., it is an automorphic 
image of x\. If we have managed to reduce an element of a free group 
(by Whitehead transformations) to an element of length 1, we immediately 
conclude that it is primitive; no further analysis is needed. 



1.3 Tietze’s Method 

Attempting to solve one of the major problems of combinatorial group the- 
ory, the isomorphism problem, Tietze introduced isomorphism-preserving 
elementary transformations that can be applied to groups presented by 
generators and relators. 

Let 



G= {x 1 ,x 2 ,... | Ri,R 2 ,...) 

be a presentation of a group G = F/R, where F is the ambient free group 
generated by xi, # 2 , • • • , and R is the normal closure of i?i, i? 2 , . . . ; i.e., the 

smallest normal subgroup of F containing ify, i? 2 > Our treatment here is 

a little different from standard in that we apply elementary transformations 
to the whole normal subgroup R and not to a particular set i?i, • • • of 
relators. In other words, even though it looks like we specify • • • 

as defining relators, we do not consider them a preferred set of elements 
whose normal closure is i?; we do this just for notational convenience. 

The elementary transformations are of the following three types. 

(Tl) Introducing a new generator : Replace (xi,X 2 >--- | i?i,i? 2 ,---) by 
{y,x i,x 2 ,... | • • • ), where S = S(x i,x 2 ,...) is an 

arbitrary element in the generators xi,X 2 , 

(T2) Canceling a generator (this is the converse of (Tl)): If we have 
a presentation of the form (y,x i,#2,... I Q, i?i, # 2 ? • • • ), where Q 
is of the form yS'" 1 , and S', i?i, i? 2 , • • • are in the group generated by 
#i, X 2 , . . . , replace this presentation by (xi, #2? • • • I i?i, i? 2 , • • • )• 

(T3) Renaming the generators : Rename generators (xi,X 2 ,...) to 
(xi x , Xi 2 , . . . ) , where i \ , i<i , . . . are arbitrary distinct indices. 

Then we have the following useful result due to Tietze (see, e.g., [227]): 

Theorem 1.3.1. Two groups (xi,x 2j ... | iii,i? 2 ,---) and (xi,x 2 ,... | 
5 i,S 2 ,.-.> are isomorphic if and only if one can get from one of the 
presentations to the other by a sequence of transformations (T1)-(T3). 

For most practical purposes, groups under consideration are finitely pre- 
sented, in which case there exists a finite sequence of transformations 
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(T1)-(T3) taking one of the presentations to the other. Still, Theorem 1.3.1 
does not give any constructive procedure for deciding in a finite number 
of steps whether one finite presentation can be obtained from another by 
Tietze transformations because, for example, there is no indication of how 
to select the element S' in a transformation (Tl). Thus Theorem 1.3.1 does 
not yield a solution to the isomorphism problem. 

However, it has been used in many situations to derive various invariants 
of isomorphic groups, most notably Alexander polynomials that turned out 
to be quite useful in knot theory (see, e.g., [85]). 

Also, Tietze’s method gives an easy, practical way of constructing 
“exotic” examples of isomorphic groups that helped to refute several 
conjectures in combinatorial group theory (see, e.g., our Section 1.4.1). 



1.4 Free Differential Calculus 

Let ZF be the integral group ring of a free group F and A its augmentation 
ideal; that is, the kernel of the natural homomorphism e : Z F -» Z. More 
generally, when R < F is a normal subgroup of F, we denote by A# the 
ideal of Z F generated by all elements of the form (r — 1), r £ R. It is the 
kernel of the natural homomorphism sr \ Z F -* Z (F/R). 

The ideal A is a free left ZF-module with a free basis {(xi — 1)}, 1 < 
i < n, and left Fox derivations di are projections to the corresponding free 
cyclic direct summands. Thus any element u £ A can be uniquely written 
in the form u = di(u)(xi — 1). 

Since the ideal A is a free right ZF-module as well, one can define right 
Fox derivatives d'^u) accordingly so that u = X^=i ( x * ~ 1 )d' i (u). 

One can extend these derivations linearly to the whole Z F by setting 
d'(l) = di( 1) = 0. 

Below we list several lemmas that we are going to need in subsequent 
chapters of the book. Most of the lemmas are standard, so we give them 
without proofs; an interested reader can find the proofs either in the original 
paper by Fox [134] or in the monograph by N. Gupta [146]. 

Lemma 1.4.1. Let J be an arbitrary left (right) ideal of ZF, and let 
u e A. Then u £ A J (u £ JA) if and only if d[(u) £ J (di(u) £ J) for 
each i, 1 < i < n. 

Let 7 k(R) denote the fcth term of the lower central series of R. 

Lemma 1.4.2. Let R be a normal subgroup of F, and let yGf. Then the 
following conditions are equivalent: 

(i) y e 7 m +iCR), m> l, 

(ii) y - 1 € A R A m , 

(iii) y- 1 € A m A fi . 
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This can be generalized to arbitrary subgroups R not necessarily normal. 
For an arbitrary i?, we let A# denote the left ideal of Z F generated by all 
elements of the form (r — 1), r G R. Then we have: 

Lemma 1.4.3. LetR be a subgroup ofF, and lety G F. Then the following 
conditions are equivalent: 

(i) y e 7m+i (R), m>l, 

(ii) y-le A m A R . 

Before we formulate the next lemma, we have to introduce some more 
notation. If R is a normal subgroup of F, then there is a familiar action via 
conjugation of the group ring Z (F/R) on the abelian group R/R' under 
which R/R' becomes the relation module for F/R. Accordingly, there is 
a map (F/R, R/R!) — > R/R ' defined by (fR,rR') -» (f~ x rf)R' and lin- 
early extended to (Z (F/R), R/R') — > R/R'. Where the meaning is clear 
from the context, we denote the elements of Z F and their natural im- 
ages in Z (F/R) by the same letters without ambiguity. Thus, for any 
v = Yhi n ifi m °d TjF(R — 1), Hi £ Z, fi G F, and r G i?, the result of 
the action of nifiR G Z(F/R) on rR' is denoted by r v R'. 

Lemma 1.4.4. Let r e R and v G ZF/Aj*. If r v = 1 (mod R'), then 
v*dr/dxi = 0 (mod Ar) for every i, where for v = Y^i n ifii v * = 

E* "i/r 1 - 

We also need the “chain rule” for Fox derivations: 

Lemma 1.4.5. Let ip be an endomorphism of F (it can be linearly extended 
to ZF) defined by (p(xk) = yk, 1 < fc < n, and let v = p(u) for some 
u, v G ZF. Then 

n 

d A v ) = ^¥>(dfc(u)) • dj{y k ), 
k = 1 

d' j {v) = Y j d' j {y k )-<p{d' k {u)). 
k = 1 

For an endomorphism <p: Xi — > yi, 1 < i < n, of the group F n , let 
J(p = (dj(yi))i<i,j<n, J!p = ( di(yj))i<i,j<n be the Jacobian matrices of p. 
We are going to need the following ramifications of Lemma 1.4.5. 

Lemma 1.4.6. If g,h G F n and h = <p(g), then for the u Fox gradients ”, 
one has 



{di{h), d n (h)) = (v?(di( 5 )), . . . , <p(d n (g))) ■ J v , 
K W, • • • , < W)‘ = J' v ' (<P(d'i(9)), • • • , 'P&m*, 

where * means transposition. 
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Lemma 1.4.7. //<£, ^ are two endomorphisms ofF n , then for the Jacobian 
matrices one has 



J io( 'll)) — Jm * Jib 






JL • W- 



Now we give a nice application of Lemma 1.4.7 due to Birman [54]; it 
is usually called the “inverse function theorem” for free groups. The proof 
that we give here is easier than the original one. 



Theorem 1.4.8 ([54]). Let (p be an endomorphism of the free group F n 
given by ip: X{ -» yi, 1 < i < n. Then yi, . . . ,y n generate the group F n if 
and only if the Jacobian matrix J ^ is (left) invertible. 

Proof. The “only if” part of this theorem follows immediately from 
Lemma 1.4.7. 

For the “if” part, recall that 

n 

V j -l = '52di(y j )(x i -l). (1.1) 

i = 1 

Now suppose that the Jacobian matrix J ^ is left invertible over Z F. From 
(1.1), we get then (in matrix form): 

Jy\Y~ 1)=X-1, 

where Y — 1 is {y\ - 1, . . . , y n - 1} written as a column, and X - 1 is 
{xi — 1, . . . , x n — 1} written as a column. That means, for some elements 
Uij G Z F n we have: 

n 

Xj - 1 = '%2u ij (y i - 1 ). 
i — 1 

In other words, every Xj — 1 belongs to the left ideal of ZF n generated 
by {yi — 1, . . . , y n — 1}. By Lemma 1.4.12 in the next section, this implies 
that every Xj belongs to the subgroup generated by {yi, . . . , y n }; i.e., Y is 
a generating set of F. ■ 



Remark 1.4.9. Since the augmentation ideal of any group ring Z p F n , 
p a prime, is a free left and right module over this ring, it is possible to 
consider Fox calculus in the rings Z p F n as well, and all technical results 
hold also in that situation. In particular, Birman’s theorem, Theorem 1.4.8, 
holds upon replacing the Jacobian matrix J ^ by its natural homomorphic 
image over Z p F n for any prime p. 

Finally, we give, without proofs, two ramifications of Theorem 1.4.8. 
The first one, due to Umirbaev [389], is a direct generalization of Birman’s 
result: 
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Theorem 1.4.10 ([389]). Let (yi, . . . , yk), 1 < k < n, be a system of 
elements of the group F n . This system is primitive if and only if the matrix 
(dj(yi)), 1 < i < k, 1 < j <n is left invertible over ZF n . 

The following ramification is due to Shpilrain [353]: 

Theorem 1.4.11 ([353]). An endomorphism X{ — » y^, 1 < i < n, 
of the group F n is a monomorphism if and only if the rows of the matrix 
Jtp = (dj(yi))i<ij< n are left ZF u -independent. 



1.4-1 Two applications 

In this section, we give two very elegant applications of free differential 
calculus. One of them was previously published, the other was not. Neither 
of the two results is new; in fact, both can be considered classical. However, 
the proofs given here supersede (by far) the original proofs in simplicity 
and elegance. Also, the ideas used in these proofs will have more serious 
applications in Parts II and III of this book. 

For the first application, we need one simple well-known fact about 
arbitrary group rings (see, e.g., [300]). 

Lemma 1.4.12. Let G be an arbitrary group , K an arbitrary commutative 
ring with unity , and KG the corresponding group ring. Let tq, . . . ,v m and 
u be elements ofG. Suppose (u — 1) belongs to the left (or right) ideal of KG 
generated by (v i — 1), . . . , (v m — 1). Then u belongs to the subgroup H of G 
generated by v i, . . . , u m . 

Proof. Let S be an arbitrary subgroup of G and u E KG an arbitrary 
element of the group ring, so that u = Yh 9i eG where c* E K. We intro- 
duce a mapping tcs on KG (a “projection”) such that tts(u) = J2 9i es 
Then it is easy to see that this mapping is AMinear and, whenever w € KS, 
one has tts(uw) = tts{u) • w, and ns(wu) = w • tts(u). 

Now let g e G, g — 1 = J2 w i ' ( v i ~ 1) f° r some Wi E KG. Apply n h to 
both sides: 

^H(g) - 1 = T H (Wi) • (Vi - 1). 

If g £ iJ, then tth (g) = 0, so that the augmentation of the left-hand side 
of this equality is —1, whereas the augmentation of the right-hand side is 0. 
This contradiction implies g E H. ■ 

We are now in a position to give the first elegant application of free 
differential calculus. Specifically, we give a simple proof of the “if” part 
of Nielsen’s “commutator test” (in a stronger form). Our notation here is 
[a, b] = aba~ 1 b~ 1 . 

Proposition 1.4.13. Let y\ and y 2 be some elements of the free group F 2 
generated by x\ and X 2 , and let R be the subgroup of F 2 generated by y\ 
and y 2 . If the commutator [xi,x 2 ] belongs to [R,R], then R = F 2 . 
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Proof. By Lemma 1.4.2, we have 

[xi,x 2 ] - 1 e AA fi . (1.2) 

Apply derivations d[ and d r 2 to both sides of (1.2): 

X^ J X 2 1 (x 2 [x 1 ,x 2 ] - 1) € Ar, (1.3) 

x 2 1 (x 2 xJ' 1 X 2 1 - 1) G A R. (1.4) 

Now (1.3) implies x 2 [xi,x 2 ] € R, and (1.4) implies x^r^x^ 1 e R, so we 
just have to show that these two elements generate F 2 . Let g = £ 2 [£i, # 2 ]? 
h = X 2 Xi 1 x 2 1 - Then hgh _1 = X 2 , so x 2 £ ii, and a?i = x 2 x gh~ x , so 
x\ £ i?, and R — F 2 . ■ 

Remark 1.4.14. We can further strengthen Proposition 1.4.13 by claiming 
only [xi,a? 2 ] £ R 2 instead of [^ 1 ,^ 2 ] € [i2, i?]. Indeed, on replacing the 
integral group ring with the ring Z 2 F, our argument remains valid because 
in Z 2 F one has r — 1 £ A A# whenever r £ R 2 . 

The second elegant application of free differential calculus concerns 
a conjecture of Magnus refuted by McCool and Pietrowski in [241]. The 
conjecture was: given two elements g, h of a free group F such that the 
one-relator groups (F \ g) and (F | h) are isomorphic, one has g conjugate 
to h by an automorphism of F. 

The counterexample constructed in [241] was very simple; it was based 
on applying a nontrivial Tietze transformation (see Section 1.3) to a given 
one-relator group. The hard part was to show that two elements were 
not conjugate by an automorphism. Of course, one can apply Whitehead’s 
method (see Section 1.2), but since there are a lot of elementary Whitehead 
transformations, this is technically rather complicated. 

We give here a simple method for detecting automorphically inequivalent 
elements that is based on counting the number of “zeros” of the Fox gradi- 
ents of given elements. The same idea plays an important role in Chapter 8 
of this book. 

Proposition 1.4.15. Let F be the free group generated by x, y , and z, 
and let g = x~ 1 y 2 x 2 z 2 , h = x~ 1 y 2 xz 2 y 2 xz 2 . Then the one-relator groups 
(F | g) and (F \ h) are isomorphic , but g is not conjugate to h by any 
automorphism of F. 

Proof. The following sequence of Tietze transformations establishes the 
isomorphism: 

(x, y, z I x~ 1 y 2 x 2 z 2 = 1) = (x, y, z \ x = y 2 x 2 z 2 ) 

- (x, y, z, u | u = x 2 , x = y 2 uz 2 ) = (y,z,,u\u = (y 2 uz 2 ) 2 ) 

= {x,y,z | x = (y 2 xz 2 ) 2 ) = (x,y,z \ x~ 1 y 2 xz 2 y 2 xz 2 = 1). 
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Now compute the abelianized Fox gradients of g and h : 

grad 0 ( 5 ) = (-z -1 + + V 2 , ® - 1 (l+ 2 /), x~ 1 y 2 x 2 (l+z)), 

grad a (/i) 

= (- X _1 + £ _ y + y 4 z 2 , x _1 (H-y)(l + xz 2 y 2 ), y 2 (\+z){l + xz 2 y 2 )). 

If we now allow x and y to take values in the group C*, we see that 
grad a (p) has no zeros, whereas grad°(h) has infinitely many zeros. 

If g was an automorphic image of h, then grad a (#) and grad°(h) would 
have the same number of zeros by Lemma 1.4.6. This contradiction com- 
pletes the proof. ■ 




2 

Test Elements 



This chapter is about test elements of various groups. The idea here is to 
distinguish, for example, automorphisms among arbitrary endomorphisms 
by means of their action on a single element, a test element. More precisely: 

Definition 2.0.1 ([352]). An element g G G is called a test element (for 
recognizing automorphisms) of a group G if, whenever p{g) = g for an 
endomorphism ip of G, it follows that (p is an automorphism. 

The condition “</?(#) = g ” in this definition can be obviously replaced by 
“< p(g ) = a(g) for some a G Aut(G)”. Thus, the definition can be informally 
rephrased as “if an endomorphism (p acts like an automorphism on one 
particular element g , then ip is itself an automorphism” . 

In this chapter, we are going to describe test elements in various groups, 
but the focus is on free groups. The first result of this particular type 
was that of Nielsen [293], who proved that if an endomorphism of the free 
group of rank 2 fixes the commutator of a pair of generators, then it is 
an automorphism. Similar results for other particular elements of a free 
group were later obtained by Zieschang [417, 418], Rosenberger [318], Dold 
[100], Rips [314], Durnev [115], and others. Shpilrain [352, 355] generalized 
most of these results and put them in perspective. Turner [383] character- 
ized test elements of a free group as elements that do not belong to any 
proper retract. Based on this characterization, Comerford [82] suggested 
an algorithm for recognizing test elements of a free group. 

The idea behind the definition of a test element has numerous rami- 
fications; for example, one can consider properties (other than being an 
automorphism) of an endomorphism that can be recovered from its action 
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on a single element. Also, one can consider test sets rather than just test 
elements. 

The situation in arbitrary groups is more complex. Turner and his stu- 
dents have several results on test elements in hyperbolic groups [289], direct 
products of groups [290], and free products of finite cyclic groups [396]. We 
concentrate here, in Section 2.1, on two-generator groups, where there were 
numerous attempts to expand Nielsen’s “commutator test” to one-relator 
groups by Hill and Pride [151], to free metabelian groups by Durnev [115], 
and to free solvable and other groups by Gupta and Shpilrain [147]. 

There is also a nice application of test elements to solving equations in 
free groups pointed out by Durnev in [116]. 

In Section 2.3 of this chapter, we describe another result related to the 
idea of test elements. D. Lee [204], based on ideas of S. Ivanov [153], 
confirmed the following conjecture of Shpilrain [42, Problem (F7)]: every 
endomorphism of a free group of finite rank is completely determined by 
its values on just two (explicitly given) elements unless the image of this 
endomorphism is cyclic. We therefore have a test set of two elements that 
allows one to distinguish any two endomorphisms of a free group that have 
noncyclic images. 



2.1 Nielsen’s Commutator Test 

Nielsen [293] gave the following commutator test for an endomorphism of 
the free group F = F 2 = {x, y) to be an automorphism: an endomorphism 
(p: F -* F is an automorphism if and only if the commutator [<p(x), <p(y)] 
is conjugate in F to [x^y^ 1 . He obtained this test as a corollary to his 
well-known result that every IA-automorphism of F (i.e., one that fixes 
F modulo its commutator subgroup) is an inner automorphism. Bachmuth 
et al. [29] have proved that IA-automorphisms of most two-generator groups 
of the type F/R! are inner, and it becomes natural to ask if Nielsen’s com- 
mutator test remains valid for those groups as well. Durnev [115] considered 
this question for the free metabelian group F/ F" and confirmed the valid- 
ity of the commutator test in this case. Here we will show that Nielsen’s 
test does not hold for a large class of F/R' groups and, as a corollary, 
deduce that it does not hold for any nonmetabelian solvable group of the 
form F/R". On the other hand, we give a sufficient condition under which 
Nielsen’s commutator test is valid for a given pair of generating elements 
of F modulo [R',F], 

The negative result for F/R' groups yields the following natural question 
(see [43, Problem (S9)]): Does the free solvable group of rank 2 and derived 
length d > 2 have any test elements? Roman’kov [317] has constructed test 
elements in the free solvable group of rank 2 and derived length 3. It is 
plausible that the same method can be used for constructing test elements 
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in the free solvable group of any bigger rank as well, but technically it is 
getting more complicated. 

We also mention a related result of Timoshenko [382] , who proved that 
a free met abelian group of rank > 2 does not have any test elements. 

Now we get to the main results of this section. 

Theorem 2.1.1 ([147]). Let R < F' be a nontrivial normal subgroup of F 
such that 

(i) the center of F/[R, F] is R/[R,F] and 

(ii) the group ring Z(F/R) is an Ore domain (i.e., Z (F/R) has no 
zero divisors and any two nonzero elements have a common nonzero 
multiple). 

Then there exist u,v £ F such that [u,v] = [x,y] (mod R'), while the 
endomorphism x — » u, y -» v of F does not induce an automorphism 
of F/R'. 

We need one preliminary lemma. 

Lemma 2.1.2. The mapping r: x xr, y -» ys with r, s £ F' defines 
an automorphism of F if and only if r = [x, /] and s = [y,f] for some 
f £ F. Similarly , the mapping r': x rx, y — )• sy with r, s £ F f defines 
an automorphism of F if and only ifr = [/, x -1 ] and s = [/, y~ x ] for some 
feF. 

Proof. Consider an automorphism r of F of the form 
r : x — » xr, y — » ys where r,sG F f . 

Then r is an IA-automorphism and hence, by Nielsen’s characterization, r 
must be an inner automorphism x —»• x^, y yf , induced by some f £ F. 
Comparing the two formulations of r yields the result. ■ 

Proof of Theorem 2.1.1. Choose some r £ R, r £ R r . Let u = r a x 
(mod R') and v = r b y (mod R') for some a, b £ Z F (to be specified later). 
Then, modulo i?', we have 

[r a x,r b ,y] = [r a x,y][r a x,r b ] y = [r a ,y] x [x,y][x,r b ] v 

= [x,y}[r a ,y] x{x ' y] [x,r b ] y = [x,y\[r a ,y\ y ~ lxy [x,r b ] y 

= [xiyWy-'XY^iX-'fy = [x,y}r^ 1 - y ^ +b ^ x ~ 1 - 1 ^ xy . 

Thus, the congruence [ r a x , r b y] = [rr, y] (mod R') is equivalent to the 
congruence 

r (a(l-y- 1 )+6(x- 1 -l)) = j ( mod 

Since Z (F/R) has no zero divisors, it follows by Lemma 1.4.4 that 
a(y -1 — 1) = b{x~ l — 1) modulo A#, 



(2.1) 
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where a,b G A^ (since R < F' implies that A r lies in A 2 F ). By 
condition (ii) of our theorem, there exist a, b £ Ar satisfying the con- 
gruence (2.1). We choose one such pair of elements a and b satisfying (2.1) 
and consider elements u and v of F such that 

u = r a x and v = r b y (modi?'). 

For this choice of u and v, it is easy to see that the congruence [u,v] = 
[x, y ] (mod i?') holds. Suppose that the mapping x — ► u, y — v induces 
an automorphism of the group F/R'. Then, this being an IA- automorphism 
of F/i?', it must be inner by a result of Bachmuth et al. [29]. In that case, 
as in Lemma 2.1.2, we would have, for some / G F, 

r a = [/, x -1 ] (modi?') and r b = [/, y~ l ] (modi?'). (2.2) 

Since r a , r b are in [i?, F] (a, b G Ap), it follows that /[i?, F] is in the center 
of F/[i?, F], which by hypothesis is i?/[i?, F]. Thus / G i? \ i?'. Now, (2.2) 
yields, by commuting appropriately with y~ x and x -1 , the congruences 

r a(y- l -V = [f, X -\y- 1 ] and = [f,y~ l , X - 1 } (mod R% 

which upon using (2.1) yield the congruence 

= [/,y _1 ,aj _1 ] (mod R’), 

or, equivalently, [/, x~ 1 y~ 1 ] = [/, y~ l x~ x ] (modi?'), since / G i?. Now, 
an application of this last congruence in the expansion of the equation 
[/, = [/, [x,y]y~ 1 x~ 1 ] modulo R’ gives [f,[x,y]] 6 R’, which im- 

plies (using Lemma 1.4.4) that [y,x] G i?, implying that F/R is abelian, 
contrary to the choice of i?. This completes the proof of the theorem. ■ 

The following corollary to Theorem 2.1.1 contrasts sharply with the cited 
result of Durnev [115]. 

Corollary 2.1.3. Let N be a proper normal subgroup of F such that F/N 
is solvable. Then there are elements u,v in F such that [u,v] = [x,y] 
(mod JV") but the endomorphism x— y v of F does not induce 
an automorphism of F/N " . 

Proof. Since F/N' is a torsion-free solvable group (see [146, p. 23], it 
follows by a theorem of Kropholler et al. [187] that the group ring Z(F/AT') 
has no zero divisors. Thus, with R = N', F/R is solvable and Z (F/R) 
has no zero divisors. It follows by a result of Lewin [214] that Z (F/R) 
is an Ore domain. Since the center of F/[i?,F] (= F/[N f ,F]) is always 
R/[R,F] (see [146, p. 117]), the hypothesis of Theorem 2.1.1 is satisfied 
by R and the corollary follows. ■ 

Corollary 2.1.4. 

(a) For each k > 3, there exists an endomorphism <pk m . F F that 
does not induce an automorphism of the free solvable group F/F^ 
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of derived length k, whereas (by Corollary 2.1.3) [ipu (%) , Vk (y)\ = 
[x,y] mod F^; 

(b) For each c > 3, there exists an endomorphism / ip c : F -» F 
that does not induce an automorphism of F/['y c (F),'y c (F)], whereas 
bPc( x )^ c (y)\ = [x,y\ mod [ 7 c (F), 7 c (F)] (by Theorem 2.1.1). 

While Nielsen’s commutator test does not hold for most F/R ' groups of 
rank 2, the groups of the form F/[R', F] behave altogether differently. Here 
we prove the following result. 

Theorem 2.1.5 ([147]). Let R be a nontrivial normal subgroup of the free 
group F = (x,y;0). If [u,v] = [x,y] ±9 mod [R',F] for some g G F, then 
u and v generate F modulo [R\F]. 

Proof. We may clearly assume that [u,v] = [x, y] mod [i?', F]. Since 
[R',F]- 1 < AfArAf (see, for instance, [146, p. 113]), we have 

[% v ] = [x, y] (mod A F A R A F )- (2.3) 

Taking right Fox derivatives of both sides of (2.3) yields 

d'u/d f x(v - [u,v]) + d'v/d'x(l - v~ x uv) = (y - [x,y]) mod ArAf , 

d'u/d'y(v - [%v]) + d , vjd t y( 1 - v~ x uv) = (1 — y~ x xy) mod A^A^. 

# (2.4) 

Now, taking the left Fox derivatives of both sides of the congruences in (2.4) 
yields four congruences modulo A# given by the following matrix equation: 

/ d'u/d'x d'v/d'x\ ( d(v — [u,v])/dx d(v — [u,v])/dy \ 
yp'u/d'y d'v/d'y J y9(l — v~ 1 uv)/dx d(l — v~ l uv)/dyj 

= / x~ 1 y~ 1 (y - 1) 1 - x~ 1 y~ 1 (x - 1)\ 

v -y~ X y~ x -y~ x x )' 

It is easily verified that the matrix on the right-hand side above is invertible 
over Z F and hence also over Z F (mod A^). Hence the Jacobian matrix 
on the left-hand side is also invertible over Z F (mod A^), which, by a re- 
sult of Krasnikov [186] (see [146, p. 29]), implies that u and v generate F 
modulo R f . It follows that u and v also generate F modulo any normal 
subgroup V of R such that R/V is nilpotent. Thus u and v generate F 
modulo [i?', F], as was to be proved. ■ 

Although the elements u and v generate the group F/[R',F] under the 
conditions of Theorem 2.1.5, we cannot, in general, conclude that the map- 
ping induced by x — > u, y — > v defines an automorphism of the group 
F/[R',F\ since, for instance, F/[R\F\ may be non-Hopfian. Since [R f ,F] 
might not be fully invariant in F, this mapping may not even define an en- 
domorphism of F/[R\ F\. As a (partial) converse of Theorem 2.1.5, we can 
prove the following result. 
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Proposition 2.1.6. Let R be a normal subgroup of the free group F such 
that R< F' and the group ring Z (F/R) has no zero divisors. If the mapping 
r: x -» u, y — )■ v induces an automorphism of the group F/[R f ,F], then 
[u, v] = [x, y] ±9 mod [R',F] for some g € F. 

Proof. Let r induce some automorphism ^ of the group F/R'. Then xj; 
is tame (Bachmuth et al. [29]) and hence a composition of inner automor- 
phisms of F/R' together with the automorphisms of F/R' induced by the 
maps x — y y, y — y x and x — > xy , y -» y. It suffices, therefore, to verify 
that the congruence [r(x),r(y)] = [x,y] ±9 mod [R',F] holds for some g 
in F when r is assumed to be an inner automorphism of F/[R',F] or else 
defined by the maps x — >> y, y -> x and x — » xy, y y. These verifications 
are straightforward, and we omit the details. ■ 



2.2 Recognizing Test Elements 

In this section, we give a remarkable characterization, due to Turner [383] , 
of test elements in a free group of finite rank along with some other interest- 
ing auxiliary results. Then, based on Turner’s characterization, we describe 
an algorithm, due to Comerford [82], for recognizing test e . unents. 

We denote by F r = F(x i, . . . , x r ) the free group of rank r >2 with basis 
{#i, . . . , x r } and by F a free group of unspecified rank. 

Definition 2.2.1 ([352]). The rank of w € F is the smallest rank of a free 
factor of F containing w. 

It is clear that if w is a test word, then w has maximal rank. The following 
example shows that the converse is not true in general. 

Example 2.2.2. Let tp: F 2 F 2 be defined by 

<p(xi) = x\x 2 x± 1 X 2 1 , <p(x 2 ) = 1. 

Then (p fixes x\x 2 xf 1 X 2 \ but x\x 2 xf X X 2 1 lies in no proper free factor 
(see, for example, [71, p. 139]). 

What distinguishes x\x 2 xf x x 2 1 from, say, 1 is that the former 

lies in a proper retract (namely, the image of ip) while the latter does not. 
We recall the definition of a retract: 

Definition 2.2.3. A retract R of a group G is a subgroup with the property 
that the identity map i: R R extends to a homomorphism p: G -> R. 
Equivalently, a retraction p : G — G is a homomorphism such that p 2 = p 
and a retract is the image of a retraction. 

Free factors are retracts, but not all retracts of free groups are free factors. 
The next definition is important for what follows. 
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Definition 2.2.4 ([152]). If p: F -» F is an endomorphism of the free 
group F, then the stable image of p is 

oo 

<P°°(F) = f) V i (F), and Voo = V >\ V °°(F). 

i = 0 

It is shown in [152] that p^ is an automorphism and clearly p°°(F ) 
contains Fix(y>), the set of words fixed by p. 

Theorem 2.2.5 ([383]). If p: F -* F is an endomorphism of the 
finitely generated free group F , then (p°°(F) is a retract of F; p°°(F) is 
a proper retract precisely when p fails to be an automorphism . If p is a 
monomorphism , then p°° (F) is a free factor. 

Proof. It follows from [231, Problem 33, p. 118] (see also [152]) that p°°(F) 
is a free factor of p n (F) for almost all n. Choose a particular such n, let F 
be a complementary free factor, and consider 

p n : F — > p n {F) = p°°(F) * F. 

If p is a monomorphism, then p n : F -* p n (F) is an isomorphism, so letting 

F' = {ip n )~ x {F), 

F = (^)- 1 (^(F)) * {<p n r\F) = <P°°(F) * F' . 

More generally, let 7 r: p n (F ) — p°°(F) be a projection on the first factor 
of the decomposition above. If p is the composition p = (p^) n on op n , 

F -^4 ip n (F) ^4 (p°°(F) <P°°(F), 

then p is a retraction. ■ 

Corollary 2.2.6. The test words in F are the words not contained in 
proper retracts. The test words for monomorphisms in F r are the words of 
rank r. 

Proof. We show that w is not a test word if and only if w is contained in 
a proper retract. It is clear that if w lies in a proper retract, then it is not 
a test word since it is fixed by the retraction that is not an automorphism. 
Conversely, suppose that w G F is not a test word and that p is an endo- 
morphism fixing w that is not an automorphism. Then, by Theorem 2.2.5, 
p°°(F) is a proper retract containing w. 

If w is not a test word for monomorphisms and is fixed by the monomor- 
phism p, that is not an automorphism, then w G p°°(F), which is a proper 
free factor; thus w has nonmaximal rank. ■ 

Example 2.2.7. The following are test elements in a free group F n : 
x\ a£, k > 2, [xi, . . . ,x n ], [xi,x 2 ] . . . [®2m-l,®2m] n = 2 m. 

In a special case n = 2, test elements can be described more explicitly. 
Let a Xi (u) be the sum of the exponents to which Xi occurs in the word u. 
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Corollary 2.2.8. A word u E F 2 , which is not a proper power , is a test 
word if and only if g. c. d.(cr Xl (u), a X2 (u)) > 1. In particular, every u E 
[^ 2 ,^ 2 ] is a test word. 

Proof. Suppose first that g.c.d.(a Xl (u),a X2 (u)) = 1. Then, upon taking 
x\ and X 2 to appropriate powers of x\, we can take u to x\ by an endo- 
morphism. Then apply another endomorphism, taking x\ to u and X 2 to 1. 
The composition of these two endomorphisms is a retraction that takes F 2 
onto the subgroup generated by u. Thus, u belongs to a proper retract of 
F 2 and therefore is not a test word by Corollary 2.2.6. 

If g. c. d .(a Xl (u),a X2 (u)) > 1, then any automorphic image of u has the 
same property. Suppose, by way of contradiction, that u belongs to a proper 
retract of F 2 . Since every proper retract of F 2 is cyclic and u is not a 
proper power, u must be a generator of a proper retract. Then, by [231, 
p. 140], upon applying an automorphism to u , we can take it to an element 
of the form x\ • c, where c belongs to the normal closure of x 2 . In an 
element v of this form, one has a Xl (v) = 1, hence g. c. d.(a Xl (v),a X2 (v)) = 
1, a contradiction. Thus, u does not belong to any proper retract of F 2 and 
therefore is a test word by Corollary 2.2.6. ■ 

For free groups of higher rank, the situation is more complex, and an 
algorithm for recognizing test elements is more sophisticated. Before we 
describe this algorithm (due to Comerford [82]), we give an example (due 
to D. Voce) showing that Turner’s characterization of test elements may 
not hold in nonfree groups. More precisely, test elements can never lie in 
proper retracts, but there might be elements that do not lie in any proper 
retract and yet are not test elements. 

Example 2.2.9. In the group K presented by (a, b | aba~ l = 6 _1 ), b lies 
in no proper retract, but the endomorphism ip defined by 

<p(a) = a s b , (p(b) = b 

is not surjective. Furthermore, (p°°(K ) = ( b ), showing that Theorem 2.2.5 
does not generalize to K. 

Now we get to Comerford’s algorithm for recognizing test elements in 
a free group. Here we shall use the following terminology and notation. We 
let G and H be free groups freely generated by sets A and B , respectively; 
when we speak of lengths of elements of H and G, it will be with respect to 
these generating sets. If a : H -» G is a homomorphism, we say that w E G 
is a Nielsen-reduced image of u E H under a if a(u) = w, u = s \ . . . su 

with si, . . . , Sk E B U 5 _1 , Sj+i ^ s~ x for 1 < i < k — 1 and, as reduced 

words on A U A~ x , a(si), . . . , a(sfc) satisfy 

o(^) ^ 1 for 1 < i < fc, 

|a(st5j + i)| > |a(si)|, |a(sj + i)| for 1 < i < k - 1, and 

|a(s»St+iSi+ 2 )| > |a(si)| - |a(si+i)| + |a(si+ 2 )| for 1 < i < k - 2. 
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In other words, no letter of u has an empty image under a, not more 
than half the image of a letter of u cancels with the image of an adjacent 
letter of tz, and no interior letter of u has its image cancelled entirely by 
the images of the two adjacent letters. 

Given a homomorphism a: H — >> G with a(u) = w, we want to produce 
an endomorphism r of H and a homomorphism /?: H G such that w 
is a Nielsen-reduced image of r(u) under (3. We use a Nielsen reduction 
technique similar to that employed by Lyndon in [225] . Following Lyndon, 
we call an automorphism of H defined by s h* st -1 with s, t G BUB -1 and 
t ^ and /: b b for b G B, b ^ s^ 1 , an elementary regular transfor- 
mation. This transformation is attached to a reduced u G H if st or £ -1 s -1 
is a subword of u. We also include the identity map on if as an elementary 
regular transformation attached to every element of H. An endomorphism 
of H defined by c 1 for some c G B and b b for b G B — {c} is called 
an elementary singular transformation and is attached to a reduced u G H 
if c or c" 1 occurs in u. A Nielsen transformation is an endomorphism r 
of H of the form r = r\ . . . r^, where each r is an elementary regular or 
singular transformation; we call r regular or singular if each factor r* is 
regular or each r* is singular. We say that r is attached to u G H if t* is 
attached to t\ . . . r^_i(u) for 1 < i < k. 

We also need the notion of the star graph (or co-initial graph or White- 
head graph) E (u) of a reduced u G H. The set of vertices of E(u) is 
B U B~ l . For each subword st of u with G B U B -1 , E(u) has a di- 
rected edge from s to £ -1 ; if p and q are the terminal and initial letters 
of u, E(u) also has an edge from p to q~ x called the external edge. For 
a subset S of BUB" 1 , the subgraph of E (u) spanned by S is a graph 
with set S of vertices and with edges being those edges of E (u) both 
of whose endpoints lie in S. We say that u G H is connected if the 
subgraph of E (u) spanned by gen(u) U (gen(u)) -1 is connected, where 
gen(u) = {b e B: b or B _1 occurs in u}. 

We shall also make use of Whitehead automorphisms of H. They are of 
two types. The first, called level Whitehead automorphisms , simply permute 
B U B~ l . The second type of Whitehead automorphism is denoted (S', s), 
where S C B\J B~ x with s G S and s -1 ^ S. The map (S', s) is defined by 
t h-» ts if t e S, t -1 ^ S, and t ^ 5, by 1 1 -» s^ts if £, t~ l G S, and by b b 
if b G B, B, B _1 ^ S, or b = 5 or s -1 . A Whitehead automorphism (S, 5 ) is 
attached to a reduced word u G H if the subgraph of E(u) spanned by S 
is connected. We observe the following: 

Lemma 2.2.10. Every Whitehead automorphism (S, s) attached to a re- 
duced word u G H is a regular Nielsen transformation attached to u. 

Proof. We use induction on |S|, the number of elements of S. If \S\ = 1, 
S = { 5 } and (S, s) is the identity map. If |S| > 2, choose t G S such that 
t ^ s but t and s are joined by an edge in E (u). It is easy to see that 
(S,s) = ({£, s},s)(S — {t},s) and that ({£, s},s) is an elementary regular 
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transformation attached to u. It remains to show that (S — {£}, s) is attached 
to g(u ), where g = ({£, s }, s). 

If S — {£} = {s}, we are finished, so suppose that p G S — {t}, p ^ s. 
Since (5, s) is attached to u , there is an edge-path e \ . . . in £(u) with e* 
an edge from pi-i to pi , pi € 5 for 0 < i < A:, po = Pi and pk = s; among 
all such edge-paths, pick one with k minimal. If p { ^ t for 0 < i < k, 
then ei . . . e/c is an edge-path in E(^(w)) witnessing that p and s are in the 
same component of the subgraph of £(p(u)) spanned by S - {£}. If Pi = t 
for some i, 0 < i < k, our choice of edge-path shows that pk-i = t and 
Pi ^ t for 0 < k — 1. In T>(g(u)) there is an edge ej b _ 1 from Pk -2 to s, and 
so e \ . . . e/c_ 2^k-i ls an edge-path from p to s in the subgraph of E(^(u)) 
spanned by S — {£}. It follows that the subgraph of E(^(ii)) spanned by 
S — {t} is connected, and so ( S — { t }, s) is attached to g(u). ■ 

We are now ready to prove the following generalization of [225, 
Proposition 6]. 

Proposition 2.2.11. If G and H are free groups freely generated by sets 
A and B, respectively , and if u E H, w E G, and a : H — )> G is a homo- 
morphism with a(u) = w, then there is a Nielsen transformation r of H 
attached to u and a homomorphism /?: H G such that r(u) is connected, 
w is a Nielsen-reduced image ofr(u) under (3, and /3(H) = a(H). 

Proof. The proof here is, for the most part, a standard Nielsen reduction 
argument. We make use of a function K introduced in [225]; K is a map 
from G to the positive integers with the properties that \u\ < |u| implies 
K(u) < K(v) and that K(u) = K(v) if and only if u = v or u = v~ x . We 
use induction, first on |gen(u)|, the number of generators of H occurring 
in u, and second on J2begen(u) H(a(b)). If u is not connected, there is some 
s 6 gen(u) such that s and s~ 1 are in different components of £(u). At 
least one of these components, say that of s, fails to contain the external 
edge. We let S be the set of vertices in the component of s in £(u) and 
note that (5, s) is attached to u , that | gen ((u)(S, s))| < | gen(u)|, and that 
a((H)(S, s) -1 ) = a(H). We replace u by (u)(S, s) and a by (5, s) _1 q; and 
appeal to the induction hypothesis. 

Suppose now that u is connected and that a(c) = 1 for some c e gen(u). 
In this case, we use the Nielsen transformation r defined by t(c) = 1 and 
r(b) = b for b G B — {c}. Now r is attached to u and | gen(r(u))| < | gen(u)|, 
so we may replace u by r(u) and again invoke the induction hypothesis. 

Finally, suppose that u is connected and a (b) ± 1 for b e gen (u) but 
that w is not a Nielsen-reduced image of u under a. Then either u has 
a subword st with s,t e B U B~ x and more than half of either a(s) 
or a(t) cancels in the product a(s)a(t), or u has a subword rst with 
r, s,t e B U B~ x and a(s) cancels entirely in the product a(r)a(s)a(t). 
As in the proof of [225, Lemma 3], there is an elementary regular Nielsen 
transformation r attached to u such that |gen(r(iz))| < |gen(w)| and 
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Ei>6gen(r(«))- f£r (( T ))) < T,be S en(u) K ( a ( b )) ■ Here we replace u by 

r(u), a by r -1 a (notice that r~ 1 a(H) = a(H)), and use the induction 
hypothesis. ■ 

Theorem 2.2.12. There is an algorithm to tell which elements of a free 
group G are test elements; that is, to decide for w G G whether or not every 
endomorphism a of G with o(t(;) = w is an automorphism of G. 

Proof. Let us fix a free generating set A for G. We may assume that A is 
finite, for otherwise no element of G is a test element. If w G G and a is 
an endomorphism of G with a(w) = w , then by Proposition 2.2.11 there 
are endomorphisms r and (3 of G such that w is a Nielsen-reduced image of 
t(w) under /?. We can effectively enumerate the pairs (y\, Pi ),..., (v 2 , (3k) 
such that w is a Nielsen-reduced image of Vi G G under the endomorphism 
/ % of G , but ^ is not surjective. (If \A\ > 2, there will be such pairs. For 
example, if A = {ai, a 2 , . . . , a n }, we have the pair (ai, /?) with (3(ai) = w 
and (3(ai) = 1 for 2 < i < n.) We next check to see if any of v \, . . . , Vk 
are endomorphic images of w in G; this is decidable by Makanin’s algo- 
rithm [232] to tell if equations have solutions in a free group. If one of 
ui, . . . , Vk is an endomorphic image of w in G, then w is not a test element 
for G; otherwise, w is a test element for G. ■ 



2.3 Test Sets 

One can consider test sets rather than just single test elements. This allows 
one to detect more subtle distinctions between different automorphisms of 
a free group. In particular, the following natural problem has attracted 
a lot of attention: 

Problem 2.3.1 ([43]). Denote by Epi(n, k ) the set of all homomorphisms 
from a free group F n onto a free group i 7 *,; n, k > 2. Are there two elements 
9 i ,92 £ F n with the following property: whenever ip(gi) = 'ip(gi), i = 1,2, 
for some homomorphisms E Epi(n, k), then p = xp? (In other words, 
every homomorphism from Epi(n, k ) is completely determined by its values 
on just two elements.) 

Addressing this problem, S. Ivanov [153] proved that every injective ho- 
momorphism from Epi(n, k) is completely determined by its values on just 
two elements. 

More recently, D. Lee [204], based on the ideas of [153], settled the prob- 
lem completely. Below we sketch some ideas from Ivanov’s paper [153] and 
refer the reader to [153] and [204] for full proofs. 

Definition 2.3.2. A nontrivial element w G F m is an M- test word for 
F m if for every endomorphism p and monomorphism xp of F m the equation 
p(w) = xp(w) implies that p is also a monomorphism. 
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Definition 2.3.3. A nontrivial word w(xi , . . . ,x n ) is a (7-test word in 
n letters for F m if for any two n-tuples (Ai , . . . , A n ), (Bi, . . . , B n ) of el- 
ements of F m the equality w(Ai , . . . , A n ) = w(B\, ^ 1 implies 

the existence of an element S G F m such that Bi = SAiS -1 for all 
i = 1 , 2 , . . . , n. 

We can make several observations related to Definitions 2. 3. 2-2. 3. 3. First, 
any (7-test word in n letters is both a test and M- test element for F n . In 
this sense, Definition 2.3.3 is the most restrictive one. The commutator 
[^1,^2] is an M-test word (for F2) but not a C-test word in two letters. If 
a (7-test word w in n letters is not a proper power, then the stabilizer of w 
in Aut F n is (t w ). 

The word w = [xi,X 2][x 3 ,X 4] • * • [^2m-i,^2m] with m > 1 is neither 
a (7- nor M-test one. To see the latter, consider an even m and put 
A\ = X \ , . . . , A m = X m , A m+ i = 2 ~ * • ? 

^2m-i = ^2m = x 2m xix^. Then rank(/4i , . . . , A 2m ) = 2 m 

but 



[Ai, A2] [A3, A4] . . . [A2m— I? A2771] — [[^1? ^2] • • • [%m— 1? ^m]? ^2m]* 

Therefore, setting ^(xi) = [xi,x 2 ] . . . [x m _i,x m ], ^(x 2 ) = x 2m , ^(x 3 ) = 
• • • = (p(x2m) = we see that w fails to be an M-test word. Clearly, 
a similar argument can be used to consider the case of odd m > 1. 

It is not clear whether or not there are M-test words in F m , m > 3, among 
the words of the form x , and this might be a rather interesting and 
difficult problem. If m = 2, then any word x\ x x ^ 2 , where k \ , k 2 are multiples 
of k > 1, is an M-test word. This follows from a result of Lyndon and 
Schiitzenberger [228] that elements of a solution of the equation x ll y l2 z l3 — 
1 with k > 1 in Fm lie in a cyclic subgroup. 

On the other hand, it is easy to see from considering values of a C-test 
word w(pc i, . . . , x n ) on a cyclic subgroup of F m that if n > 1 then w G F„, 
where F„ is the commutator subgroup of the free group F n with basis 
{xi, . . . , x n }. This implies that there are no (7-test words among the words 

rr k 1 rrkm 

0^1 ... 0/ m . 

The words [xi,...,x m ] with m > 2 are neither M- nor (7-test ones. 
To see this, put Ai = X3X1X3 1 , A 2 = x 3 x 2 Xg 1 , A 3 = [xi,x 2 ]. Then 
rank(Ai, A 2 , A3) = 3 and 

pi,/4 2 ],/4 3 ] = [a; 3 [xi,x 3 ]a; 3 1 ,[a: 1 ,X 2 ]] = [[x 3 , [xi,x 2 ]], [xi,x 2 ]]. 

Hence, upon setting (p(xi) = x 3 , (p{x 2 ) = <p(x 3 ) = [xi,x 2 ], we see that 
[[xi , X2] , x 3 ] is not an M-test word, and now the claim becomes obvious. 

Thus, Definitions 2. 3. 2-2. 3. 3 are rather restrictive, and the very existence 
of M-test words for F m , m > 2, and, especially, (7-test words in n letters 
for F m with m, n > 1 is unclear. However, it turns out that (7-test words 
do exist. 
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Theorem 2.3.4 ([153]). For arbitrary n > 2, there exists a word 
w n (x i, . . . ,x n ) that is a C-test word in n letters for any free group F m 
of rank m> 2. In addition , w n (x i, . . . , x n ) is not a proper power. 

Corollary 2.3.5. There is an element u G F m such that if p is an endo- 
morphism , ip is a monomorphism of F m , and p(u) = ip(u), then p is also 
a monomorphism and, more specifically, p = rfy, where r u is the inner 
automorphism of F m defined by means of u and k is an integer. 

Corollary 2.3.6. There are two elements u\,u 2 G F m such that any 
monomorphism \p of F m is uniquely determined by 'ipiui), ip(u 2 ). 

Corollaries 2.3.5 and 2.3.6 follow immediately from Theorem 2.3.4: 
It suffices to put u = w m (xi, . . . ,x m ) in Corollary 2.3.5 and u\ = 
w m (xi, . . . ,x m ), u 2 i (w m (x 1 , . . . ,x m )) in Corollary 2.3.6. 

The construction of the C-test word w n (x i, . . . , x n ) is as follows: If n — 2, 
then 



W 2 (x 1 ,x 2 ) = [a^, 2|] 100 a:i [xf , a;^] 200 xi [x^, x^] 300 ^! 1 [x^,^] 400 

.8 „8l500 



x x ^ lxl ^] 500 x 2 [x« ) x«] 600 x 2 [x«, x«] 700 o ; 2 - 1 K, 



For n > 3, 

W n (x l, . . . ,X n ) = W 2 (w n - l(xi,. . . ,X n -i),W n -i(x 2 , . . . , X n )) . 

To conclude this section, we give a definition of the test rank of a finitely 
generated group G: 

Definition 2.3.7. Let G be an n-generator group. Call a set of elements 
{< 7 i, . . . ,#*;}, k < n, a test set for the group G if, whenever p(gi) = gi , 
i = 1, . . . , fc, for some endomorphism / of the group G, this p is actually 
an automorphism of G. The test rank of G is the minimal cardinality of a 
test set. 

A natural question now is (see [43, Problem (FP23)]): can the test rank 
of G be equal to 2 if n > 2? If G has a test element, the test rank of G is 
obviously 1. For example, any free group of finite rank has test rank 1. On 
the other hand, there are groups (for example, free abelian groups of finite 
rank) whose test rank equals their rank. (Obviously, it cannot be bigger 
than that.) 

Timoshenko [382] proved that a free metabelian group of rank > 3 has 
test rank 2. Rocca and Turner [316] have shown recently that for any pair of 
integers ( k , n) with 1 < k < n, there are finitely generated abelian groups 
of rank n and test rank k. 
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2.4 The “Double Jacobian” Matrix 

In this section, we establish a direct connection between the “test element” 
approach to recognizing automorphisms and the “inverse function theorem” 
due to Birman [54] (see our Section 1.4). 

We define for any element u £ F n the “double Jacobian” matrix D u = 
(d^(dj(u)))i<^j< n , where dj is the “usual”, or left, Fox derivation and d \ 
is the right Fox derivation (see Section 1.4). Then: 

Theorem 2.4.1. Let p be an endomorphism of the group F. It is 
an automorphism 

(i) if and only if the matrix D^ u ) is invertible over Z F with u = 
[xi,x 2 ] . . . [x n -i,x n \, n even; 

(ii) if and only if the natural image over Z 2 F of the matrix D v? ( u ) is 
invertible over Z 2 F with u = x\x\ . . . x\. 

Proof, (i) Let p(xi ) = y*, 1 < i < ra, n — 2m, and let v = p(u). Apply 
a left Fox derivation dj to both sides of this equality; by Lemma 1.4.5, this 
gives 

d i( u )= ^2 v(dfc(«))di(yifc). (2.5) 

l<fc<n 

Note that every p(dk(u )) has augmentation 0 since u belongs to the com- 
mutator subgroup F* so that dk(u) € A, 1 < k < n. Applying now a right 
Fox derivation d' to both sides of (2.5), we get by Lemma 1.4.5 

d i(dj(v)) = ^2 XI d-(ym)^(d^(4(«)))dj(l/fe). (2.6) 

1 <k<n 1 <m<n 

When i and j run through {1 ,...,n}, (2.6) becomes a system of 
n 2 equalities that can be written in the matrix form as 

D<p(u) = (2.7) 

where J \p = {dj(yi))i<ij< n is the left Jacobian matrix of (p\ J ^ = 
(di(yj))i<i,j<n is the right Jacobian matrix of ip. 

Suppose the matrix L^( n ) is invertible. Then every matrix on the 
right-hand side of (2.7) must be invertible, too, which implies <p is 
an automorphism by Birman’s result. 

Conversely, suppose p is an automorphism. Then, again by Birman’s the- 
orem, the matrices J ^ and J ^ are invertible (actually, the result of Birman 
applies to the matrix J^, but applying Lemma 1.4.7 immediately gives the 
same result for J * as well). Now we have to consider the matrix D u \ it is 
a block-matrix having m 2x2 matrices Bk along the diagonal and zeros 
below; here 

B k _ f X 2k-l X 2k( X 2k — 1) 1 — x 2k-l x 2k( x 2k-l ~ 

V ~ X 2k x 2k~ x 2k x 2k-l J’ 
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1 < k < m. It can be easily verified that every matrix Bk has the inverse 
B -1 = f X2k-1 - xlk -1 (! - x 2k-l){x2k - 1) - X 2 k 

k V x 2k-l X2k-1 

Hence the matrix D u is invertible, and so is the matrix on the 

left-hand side of (2.7). This completes the proof of part (i) of the theorem. 

(ii) The proof goes along the same lines as that of part (i) upon re- 
placing the group ring Z F with Z 2 F. On the right-hand side of (2.5), the 
elements (p(dk(u)) will have augmentation 0 because, in the ring Z 2 F, one 
has dk(u) G Ai? whenever u G F 2 . Then, Birman’s theorem remains valid 
on replacing Z F with Z 2 F (see Remark 1.4.9). Finally, for u = x\x\ . . . x\, 
the matrix D u is an upper triangular matrix with the units on the diagonal; 
in particular, it is invertible over any ring, and this completes the proof. 



It is easy to produce many examples of elements u of the group ring Z F 
with D u invertible. However, in the group F itself, the only elements we 
know with this property are [xi, x 2 \ . . . [x n _i, x n \, . . . x\ and their au- 
tomorphic images. We have to admit that we do not know what makes the 
double Jacobian matrix of an element u G F invertible. It is clear that if 
an element u G F has the matrix D u invertible, then it is a test element 
for recognizing automorphisms; the converse, however, is not true. For ex- 
ample, elements of the form x\x\ . . . p > 3, are test elements, but they 
have the double Jacobian matrix noninvertible. 

Remark 2.4.2. If we switch left and right Fox derivatives in the definition 
of the double Jacobian matrix (i.e., put D' u = (di(dj(u)))i<ij< n ), then D' u 
turns out to be the transpose of D u . In particular, Theorem 2.4.1 holds 
also on replacing D^u) wittl D ^( u )- 




3 

Other Special Elements 



In this chapter, we consider various generalizations of primitive elements: 
generic elements in the sense of Stallings [375] , almost primitive elements 
in the sense of Rosenberger [318], and A-primitive elements in the sense of 
Shpilrain [356]. It turns out that all these generalizations are related, but re- 
lations between them are rather nontrivial, which opens several interesting 
research avenues. 



3.1 A-Primitive Elements 

We start with A-primitive elements. These are defined by means of ho- 
mological properties of the corresponding one-relator group. They make 
“nice” orbits under the action of the automorphism group of a free group 
and also have other interesting properties that bring together group theory, 
topology, and homological algebra. 

As usual, let F = F n be the free group of a finite rank n > 2 with a set 
X = {xi, . . . , x n } of free generators. 

Definition 3.1.1 ([356]). Let J be a right ideal of the free group ring Z F. 
An element u E F is called J-primitive if Fox derivatives (see Section 1.4) 
of the element u generate J as a right ideal of Z F. 

Now “usual” primitive elements are the same as ZF-primitive ones — 
this nontrivial fact follows from a matrix characterization of primitive 
elements given by Umirbaev [389] (see also [95, Corollary IV.5.3]). 
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We focus here on A-primitive elements of a free group F, where A is the 
augmentation ideal of Z F. Another motivation for considering A-primitive 
elements comes from commutative algebra. We recall some facts here very 
briefly. A vector (pi, . . . ,p m ) of (Laurent) polynomials is called unimodular 
if pi , . . . , p m generate the whole (Laurent) polynomial algebra P as an ideal. 
Then, a vector (pi,...,p m ) of Laurent polynomials is called A -modular 
(see [20]) if pi, . . . ,p m generate the augmentation ideal A as an ideal of 
the whole algebra. Suslin [379] and Artamonov [20] have proved that the 
group GL m (P), m > 3, acts transitively on the set of all unimodular and 
all A-modular vectors, respectively. 

Now the result of Umirbaev cited above can be considered a free group 
analog of Suslin’s result: it says that the group Aut(F) acts transitively on 
the set of all ZF-primitive elements (i.e., of those with a unimodular vector 
of Fox derivatives). 

The desire to get a free group analog of Artamonov’s result draws our 
attention to A-primitive elements. First we note that one can give an 
equivalent definition of A-primitivity using a different language: an ele- 
ment u G F is A-primitive if and only if the cohomology group H 2 (G, Z G), 
when regarded as a G-module (with G-action induced by the right regu- 
lar action on ZG), is isomorphic to the trivial G-module Z, where G is 
a one-relator group (F | u). 

If the group F has an even rank n = 2m, then the element 
tl„ = [Xl,X 2 ][x 3 ,X 4 ] ■ • • [x„_i,X n ] 

is A-primitive (and hence so is every element from OrbAut (F)( u n))- On 
the other hand, a free group of an odd rank has no A-primitive elements 
whatsoever — see Corollary 3.1.6 below. R. Bieri has pointed out to us 
that a combination of results of [53], [117], and [417] yields the following: 

Theorem 3.1.2. In a group F 2m ? m > 1, any A-primitive element is an 
automorphic image o/u 2m = [xi,x 2 ] [x 3 , £4 ] . . . [:r 2m _i, #2m]* 



This, together with Corollary 3.1.6 below, gives a free group analog of 
Artamonov’s result: 

Corollary 3.1.3. In a free group F of finite rank, the group Aut (F) acts 
transitively on the set of all A-primitive elements. 

In the case of F 2 , Theorem 3.1.2 yields an explicit combinatorial descrip- 
tion of A-primitive elements since every automorphic image of [#i,x 2 ] in 
the group F 2 has the form [x\,X 2 ] 9 or [x 2 ,xi] 9 for some g G F 2 . Note that 
there is also an explicit combinatorial description of ZF 2 -primitive elements 
due to [71] (see also [296]). 

There is also a nice matrix characterization of A-primitive elements: 
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Proposition 3.1.4. 

(i) An element u £ [F n ,F n ] is A-primitive if and only if the matrix 
D u = {d'j(di(u)))i<ij< n is invertible over the ring Z F n . 

(ii) If cp takes a A-primitive element of the group F n to another 
A-primitive element , then p is an automorphism. 

The matrix D u (the “double Jacobian” matrix) was introduced in [355] 
(see also our Section 2.4). Here d' denotes the right partial Fox derivation 
with respect to x^, whereas di is the “usual”, left Fox derivation — see 
Section 1.4. 

We also treat the case of a free metabelian group M n = F n /F\ to obtain 
Corollary 3.1.6 below. (In the definition of a A-primitive element in M n , 
we consider abelianized Fox derivatives.) 

Theorem 3.1.5. 

(i) The group Aut(M 2 ) acts transitively on the set of all A-primitive 
elements of M 2 - More specifically , every A-primitive element of M 2 
is an automorphic image of the element [xi,X 2 ] and therefore has the 
form [xi,X 2 \ 9 or [x 2 ,xi] 5 for some g £ M 2 . 

(ii) There are no A-primitive elements in the group M n if n is odd. 

Corollary 3.1.6. There are no A-primitive elements in the free group F n 
if n is odd. 

Talking about a general case of J-primitive elements for an arbitrary 
right ideal J of the group ring ZF, we note first of all that J-primitive 
elements do not always exist. For example, if J cannot be generated by less 
than (n + 1) elements, then there are obviously no J-primitive elements 
in the group F n . In fact, if g is a J-primitive element and the ideal J is 
fc-generated, then g has outer rank k in the sense defined in [352] (the 
minimum number of free generators on which an automorphic image of g 
can depend). This follows from a result of Umirbaev [391]. 

Deciding for which (right) ideals J of the group ring Z F the group 
Aut(F) acts transitively on the set of all J-primitive elements of F seems 
to be a difficult problem. Actually, it makes sense only for characteristic 
ideals (i.e., those invariant under free group automorphisms). Note that 
if an element g £ F is J-primitive for a characteristic right ideal J, then 
every automorphic image of g is J-primitive, too. 

Also, it is easy to show that for any m-generator right ideal J of ZF, 
the group GL m (ZF) acts transitively on the set of all m-tuples of elements 
generating J. 

Of particular interest are J-primitive elements of maximal outer rank n = 
rank F in the case where J is the right ideal generated by Fox derivatives 
of a single group element: 
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Proposition 3.1.7. Let g £ F n be an element of outer rank n, and let J 
be the right ideal ofZF n generated by Fox derivatives of g. If a monomor- 
phism (i.e., injective endomorphism) (p of the group F n takes g to another 
J -primitive element , then ip is actually an automorphism . 

In particular: 

Corollary 3.1.8 (see [383]). Let g £ F n be an element of outer rank n. 
If if(g) = g for some monomorphism ip of the group F n , then ip is actually 
an automorphism. 

Now we get to the proofs of the results of this section. 

Lemma 3.1.9. Let J be a right ideal ofZF ( hence a free right module over 
Z F) generated as a free module over Z F by ui, . . . , u m . Then the following 
conditions are equivalent: 

(a) A matrix M = (o>ij)i<i,j< m is invertible over Z F (i.e., M £ 
GL m (ZF)); 

(b) The elements yj = — 1 )o>kj, 1 < j generate the ideal J 

as a right ideal ofZF. 

Proof. Suppose M is invertible over Z F; denote by U the row matrix 
and by Y the row matrix (yi, . . . ,y m ). Then C/MM -1 = 
YM~ l = 17, which means that ui, . . . , u m belong to the right ideal of Z F 
generated by yi, . . . ,y m . Conversely, suppose we have YB — U for some 
matrix B over Z F. Then UMB = U, and hence MB = /, the identity 
matrix, because (ui, . . . , u m ) form a free basis of a free right ZF-module J. 
This implies M £ QL m (ZF) — see, e.g., [80]. ■ 

Before we get to a proof of Theorem 3.1.2, we consider a couple of 
examples. 

Example 3.1.10. The element u — [#i,# 2][#3>#4] • • • [#2m-i>^2m] of the 
group F2m is A-primitive since the corresponding double Jacobian matrix 
D u is invertible — see our Section 2.4. 

Example 3.1.11. The element v = [^1,^2] [^2 ,£3] [^3 ,£4] of the group F4 
is A-primitive. It is not quite obvious that v is an automorphic image of 
u = [xi , X2] [xs , £4] . However, this is the case: u is taken to v by the following 
automorphism: x\ — > xix^ 1 ; X2 — » X3X2X3 1 ; £3 -4 £3; £4 -4 £4. 

Proof of Theorem 3.1.2. We give a proof here without introducing 
background material, just referring to [53] and [117] for details. 

First of all, it is an immediate consequence of the definition that g £ F 
is a A-primitive element if and only if for the right ideal J g of the group 
ring ZF generated by Fox derivatives of u one has the factor-module Z Fj J g 
isomorphic to the trivial F-module Z. In other words, as we have mentioned 
before, the cohomology group JT 2 (G,ZG) of the one-relator group G = 
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(F | g) is infinite cyclic. It is also clear that the group G is torsion-free for 
if it was not, the element g would be a proper power, and such an element 
could not be A-primitive since the augmentations of its Fox derivatives 
would have a nontrivial common divisor. 

Then, Theorem 9.3 of [53] implies that the group ( F | g) is a Poincare 
duality group of dimension 2 (PD 2 -group). It has been proved in [117] that 
every one-relator (torsion- free) PD 2 -group is a surface group. 

Applying now a result of Zieschang [417], we see that, when F = F 2 m , 
g must be an automorphic image of U 2 m = [#i, ^ 2 ] [^ 3 , £ 4 ] . . . [# 2 m-:b # 2 m]- 
This completes the proof. ■ 

Proof of Proposition 3.1.4. 

(i) First suppose the matrix D u = ( dj(di(u)))i<ij< n is invertible. Then, 
by Lemma 3.1.9, the elements yi = J2k=i( x k ~ 1 )d / k (di(u)) = di(u) — 
e(di(u)), 1 < i < n, generate the ideal A as a right ideal of Z F. Since 
u G F f , we have e(di(u)) = 0, 1 < i < n, by Lemmas 1.4.1 and 1.4.2. 
Therefore, u is A-primitive. 

Conversely, if u is A-primitive, then the elements di(u) = Ylk=i( x k ~ 
1 )d' k (di(u)) generate A as a right ideal of Z F. Again by Lemma 3.1.9, the 
matrix D u is invertible. 

(ii) We use an argument from [355] implying that if a matrix D^ u ) is 

invertible and u £ F f , then ip is an automorphism. Applying part (i) of this 
proposition completes the proof. ■ 

Proof of Theorem 3.1.5. First of all, note that if u is A-primitive, then 
u £ M'; otherwise some di(u) wouldn’t belong to A by Lemmas 1.4.1 and 
1.4.2. 

(i) Let h G M^; then we can write h as [xi,X 2 ] w for some w G ZA 2 = 
r L(M 2 /M! 1 ). Then, for abelianized Fox derivatives (we denote them the same 
way as the ones in a free group ring when there is no ambiguity) we have 
di(h) = w-di([x 1 ,^ 2 ]). Hence, if di(ft), i = 1, 2, and generate the same ideal 
of ZA 2 as di([x 1 , 2 : 2 ]), fh e element w should be invertible in Z A 2 , which 
means it has the form ±g for some g G M 2 /M 2 . 

Thus h = [#i, X 2 ] ±9 ; in particular, h is an automorphic image of [#i, x^. 

(ii) Consider basic commutators of weight 2 in the group M n : c\ = 

[x!,x 2 ], c 2 = [xi,x 3 ],. . . , c N = [x n -i,x n ], where N = n(n - l)/2, and 
consider a product of the form w = c\ x (%*... . Evaluate abelianized 

Fox derivatives of the element w: 

d\(w) = ki(x 2 - 1) + • • • + K- i(x n - 1); 

d 2 {w) = -ki(xi - 1) + k n (x 3 - 1) H + k 2n -3( x n ~ 1); 

dz(w) = -k 2 (x 1 - 1 ) - k n (x 2 - 1 ) + ^ 2 n— 2(^4 -!) + ••• + k 3n - 6 (x n - 1 ); 



dn(uj) k n —\(x\ 1) * ^n(n— l)/2( x n— 1 I)* 
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We are now going to show that these derivatives do not generate A even 
modulo A 2 . To do that, it suffices to show that they are linearly dependent; 
i.e., that the nxn matrix of coefficients (its (i,j) th entry is the coefficient 
at ( Xj — 1) in the decomposition of di(w) above) has determinant 0. 

It is easy to see that this matrix (denote it by A = ( )) is antisym- 
metric, with zeros on the diagonal. The determinant of a matrix like that 
must be 0 if n is odd. Indeed, consider a summand . . . a n ^ n in the 

decomposition of the determinant. If there is at least one diagonal element 
among these ak,i k , then the product is 0. If all ak,i k are off-diagonal ele- 
ments, consider the “reflection” • • • a i n ,n- These two summands go 

with different signs since = —aji and n is odd. Therefore, they cancel 
out, which proves that the determinant of A equals 0. This completes the 
proof of part (ii). ■ 

Proof of Corollary 3.1.6. If there were a A-primitive element in F n , its 
image in M n would be a A-primitive element of M n , which contradicts 
Theorem 3.1.5 (ii). ■ 

Proof of Proposition 3.1.7. Let h = ip(g) be a J-primitive element. 
Since g has outer rank n, the right ideal J is n- generated by [391]. It 
follows that the elements di(p), . . . , d n (g) freely generate J as a right ideal 
of Z F n . Indeed, if there were a (right) ZF n -dependence between these ele- 
ments, then one of them would belong to the right ideal generated by the 
others — this follows from a general theory of [80]. Therefore, J could be 
generated by less than n elements. 

Thus, by Lemma 3.1.9, for some matrix M G GL n (ZF n ), we have 

{di(h ), . . . , d n (h)) = (di(ff), . . . , d n (g))M . (3.1) 

On the other hand, by the “chain rule” , we have 

(di{h), . . . ,d n (h)) = (ipidxig)), . . . ^(dnig)))^. 

This together with (3.1) gives 

(di(g),...,d n (g)) = (ip(di(g)), . . . ^(dnig^J^M- 1 . (3.2) 

This means J C tp(J). Suppose J C ip(J) (i.e., J ^ Since (p is 

a monomorphism, this implies (p k (J ) C p k+1 (J) for any k > 1 so that we 
have an infinite ascending chain of n-generated right ideals 

J C ip(J) C ip 2 {J) C ..., (3.3) 

which is impossible. Indeed, suppose we have two n-generated right ideals 
Ji and J2 with (free) bases {ai, . . . , a n } and { 61 , ... , b n }, respectively, and 
suppose that J 2 C Ji. We can assume that the base of J\ is reduced in 
the following sense. The free group ring Z F is known to be graded (see, 
e.g., [80]), so we can consider the leading part (or the leading monomial) 
di of every ai with respect to this grading. Now we say that our base is 
reduced if di = dj • u for some u G Z F implies i = j. 
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If our base was not reduced, we could reduce it in a finite number of 
steps upon replacing a* with -a,j-u every time we have a* = aj • u, i ^ j. 

Thus, we assume that the base of J\ is reduced, and then we see that 
since every bi is of the form ^27= l a i' v ii the sum °f weights of the leading 
parts bi of all bi is no smaller than that of the leading parts of a* (here we 
use the fact that in a free ideal ring one cannot have • u = aj • v unless 
a,i-u = aj or = aj • v — see [80]). 

Since we can have only finitely many different right ideals whose bases 
have a given sum of weights of the leading parts of their elements, this 
proves that (4) is impossible. 

Thus, we should actually have J = <p(J), in which case the matrix 
J^M -1 on the right-hand side of (3.2) must be invertible by Lemma 3.1.9. 
Therefore, J ^ is invertible, too, and hence cp G Aut(F n ) by [54]. ■ 

Corollary 3.1.8 follows immediately. 



3.2 Almost Primitive and Generic Elements 

Our exposition here follows [132]. 

Definition 3.2.1 ([132]). An almost primitive element (APE) is an ele- 
ment of a free group F that is not primitive in F but is primitive in any 
proper subgroup of F containing it. 

This can be extended to arbitrary groups in the following manner. An el- 
ement g G G is primitive in G if g generates an infinite cyclic free factor 
of G\ i.e., g has infinite order and G = (g) * G\ for some G\ C G. g is then 
an APE if it is not primitive in G but primitive in any proper subgroup con- 
taining g. Rosenberger [319] proved that in the free group F = F(xi , ?/*, Zj), 
1 < i < m, 1 < j < n, of rank 2m + n, the element 

[x u yi] ■ • -[x m ,y m ]2^ 

where the pi are not necessarily distinct primes, is an APE in F. Rosen- 
berger [319] also proved, in a different setting, that if A, B are arbitrary 
groups containing APEs a, 6, respectively, then the product ab is either 
primitive or an APE in the free product A * B. This was reproven by 
Brunner, Burns, and Oates-Williams [60], who also prove the more difficult 
result that if a and b are tame APEs in groups A, B, respectively, then their 
product normally is a tame APE in A*B. An APE w in a group G is a tame 
APE if whenever w a G H C G with a > 1 minimal, then either w a is prim- 
itive in H or the index [G : H] is a. It follows easily that [a±, b\\ . . . [a g , b g \, 
g > 1, is a tame APE in the free group on ai, &i, . . . , a^, b g . We note that 
Brunner, Burns, and Oates-Williams give a more technical definition of 
a tame APE. 
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Definition 3.2.2. Let Wbea variety defined by a set of laws V. (We refer 
to the book of H. Neumann [291] for relevant terminology.) For a group G, 
we let V(G) denote the verbal subgroup of G defined by V. An element 
g £ G is U-generic in G if g £ V(G) and whenever H is a group, / : H G 
a homomorphism, and w = f(u) for some u £ V(H), it follows that / is 
surjective. Equivalently, g 6 G is ZV-generic in G if g £ V(G) C G but 
g ^ V(K) for every proper subgroup K of G [375]. An element is generic 
if it is U- generic for some variety U. 

For n = 0 we have U n = ^4, the abelian variety. Stallings [375] and 
Dold [100] have given sufficient conditions for an element of a free group 
to be U n -ge neric. Using this, it can be shown that x™ x 1 ^ . . . x 7 ^ is U n - 
generic in the free group on aq, . . . ,x m for all n > 2, and if m is even, 
then [#i, # 2 ] • • • [x m _i, x m ] is 74-generic in the free group on ri,...,x m for 
n = 0 and for all n > 2. These facts are also consequences of a result of 
Rosenberger [318]. 

Comerford [82] points out that if G is Hopfian, which is the case if G 
is free, then being generic implies being a test word (see our Chapter 2). 
Thus, for free groups we have 

generic — > test word. 

Comerford also shows that there is no converse. In particular, he shows 
that in a free group of rank 3 on x,y, z the word w = £ 2 [y 2 ,z] is a test 
word but is not generic. We can also show that, in general, generic does not 
imply APE. Suppose F = F(x,y) is the free group of rank 2 on #, 2 /, and 
let w = x 4 y 4 . Then w is 74-generic but w is not an APE since w £ (x 2 , y 2 ) 
and is not primitive in this subgroup while this subgroup is not all of F. 

Further, in general it is not true that being an APE implies being a test 
word. Again let F = F(x,y) and let w = x 2 yx~ 1 y~ 1 . Brunner, Burns, and 
Oates-Williams show that w is an APE. However, Turner [383] shows that w 
is not a test word. Since generic elements are test words in a Hopfian group, 
this example shows further that APE does not imply generic in general. 
This is really to be expected since test words are strongly nonprimitive. 
However, Theorem 3.2.3 below shows that many APEs are indeed generic 
and therefore test words. 

Recall that a variety U defined by the set of laws V is a nontrivial variety 
if it consists of more than just the trivial group. In this case V(jF) ^ F for 
any free group F. 

Theorem 3.2.3. Let F be a free group and B a nontrivial variety defined 
by the set of laws V. Let w £ V(F). If w is an APE , then w is B-generic. 
In particular , w is a test word. 

Proof. Let w £ V(F) be an APE and let ip: H — >► F be a homomorphism 
with ip(u) = w for some u £ V(H). As in the statement of the theorem, V is 
the set of laws defining the nontrivial variety B. Let if be a proper subgroup 
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of F. If w £ K , then clearly w £ V(K). If w € if, then since w is an APE, 
w is primitive in K since if is a proper subgroup of F. Furthermore, since 
B is a nontrivial variety and K is free, we have that K ^ V(if). It follows 
then from the primitivity of w in K that w £ V(K). Therefore w 6 V(F) 
and for any proper subgroup K of F we have w £ V(if) and hence w is B- 
generic. Since free groups are Hopfian, w must then be a test word. ■ 

In particular, let F(n) be the subgroup of the free group F generated by 
all commutators and nth powers, n > 2 or n = 0; that is, F(n) = V n (F). 
Then: 

Corollary 3.2.4. Let w € F(n) with n > 2 or n = 0. If w is an APE, 
then w is U n -generic and w is a test word. 

Now we give, without a proof, a class of examples of generic elements in 
F 2 . We refer to [132] for details. 

Theorem 3.2.5. Let F be a free group on a , b and let X = {x \, . . . , Xk), 
k >1, be a finitely generated subgroup of F. Suppose that X contains the 
element [a n , b m ] for positive integers n, m. Then {x\, . . . , Xk} can be carried 
by Nielsen transformations into a free basis {yi, . . . ,y p }, 1 < p < k, for 
which one of the following cases occurs. 

(1) yi = [a n ,b m \ is a primitive element of X; 

(2) yi = a a , 1 < a < n, a\n and 
V 2 = b 0 , 1 < (3 < m, /3\m; 

(3) y\ = a a , 1 < a < n, a\n and 
y2 = b m a (3 b - m , 1 < 0 < n, (3\n ; 

(4) yi = b a , 1 < a < m, a\ m and 
y 2 = a n b^a~ n , 1 < (3 <m, (3\m; 

(5) yi = a a , 1 < a < n, a\n and 
y 2 = b m a (3 , 1 < j3 < a; 

(6) y 1 = b a , 1 < a < m, a\m and 
y 2 = aP'bP , 1 < j3 < a; 

(7) yi = a n b m , y 2 = a a , 1 < a < 2 n, a\2 n and 
ys = b@ , 1 < /? < 2m, (3\2m. 

Using this theorem, we obtain the following corollaries. 

Recall that U n is the variety generated by the laws V n = {{x,y),z n }, 
n = 0, or n > 2. We let C n be the variety generated by the laws W n = 
{[x n ,y n ]}, n > 1. We then obtain the following class of generic elements. 

Corollary 3.2.6. Let F be a free group of rank 2 on x,y. Then [ x n ,y n ] is 
Ln-generic in F , but for n > 2 it is not U n -generic in F. 
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Corollary 3.2.7. Let F be a free group of rank 2 on x , y. Then the element 

[x n , y m ], n, m > 1, is an APE if and only if n = m = 1. 

Recall that in general it is not true that being an APE implies being 
a test word. As mentioned earlier, if F — F(x,y) and w = x 2 yx~ 1 y ~ x , 
then w is an APE but is not a test word. Since generic elements are test 
words, this example shows further that APE does not imply generic in 
general. However, using the same techniques as in Theorem 3.2.5, we can 
generalize the fact that the element w above is an APE to obtain further 
examples of APEs and test words. 

Theorem 3.2.8. Let F = (a, 6;} and let X = (xi,...,Xk) C F, k > 1. 
Suppose a n ba~ 1 b ~ 1 G X, n > 2. Then {xi,...,Xfc} can be carried by 

Nielsen transformations into a free basis {yi, . . . 1 < P < k, of X 

such that one of the following cases holds: 

(1) yi = a n ba~ 1 b ~ 1 or 

(2) yi =a, y 2 = b. 

From this theorem and Theorem 3.2.3, we get the following corollary: 
Corollary 3.2.9. Let F = (a, 6;). Then 

(1) a n 6a~ 1 6“ 1 , n > 2, is an APE ; 

(2) a n 6a“ 1 6 _1 , n>3, is U n -\-generic. 




4 

Automorphic Orbits 



Automorphic orbits in a free group F n of finite rank are sets of the form 
OrbAutF n (^) = £ F n , v = <p(u) for some <p £ Aut(F n ) and a fixed 

u £ F n }. One special and very interesting automorphic orbit is the set of 
all primitive elements (these are automorphic images of a free generator 
of F n ). In particular, the following problem, along with its generalizations 
[352], has been a source of inspiration for several people: 

Problem 4.0.1 ([352, 43]). If an endomorphism <p of a free group F n 
takes every primitive element to another primitive, is (p an automorphism? 

Recently, D. Lee [205] has settled this problem in the affirmative for 
every n. Then, in [206], she settled a more general problem: 

Problem 4.0.2 ([352]). If an endomorphism ip of F n preserves an auto- 
morphic orbit OrbAutF n (^) 5 u ^ 1; i.e., if ip acts “like an automorphism” 
on one particular orbit, then is <p itself an automorphism? 

We refer to the original paper [205] for a rather lengthy argument that 
settles Problem 4.0.1 and reproduce below, in Section 4.3, an elegant 
argument that settles Problem 4.0.2. 

Other interesting problems arise from considering “abridged” orbits 
{v £ F n ; |u| = |u|, ip(v) = u for some (p £ Aut(F n )}. For instance, the 
problem of counting primitive elements of a given length (see Problem (F17) 
in [43]) has received considerable attention lately. 

Another interesting (and important) problem is the following: 
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Problem 4.0.3 ([43]). Let u be an element of the free group F n whose 
length \u\ cannot be decreased by any automorphism of F n . Let A(u) denote 
the set of elements {v £ F n \ |v| = \u\, f(v) = u for some / £ Aut(F n )}. 
How fast does the cardinality of A(u) grow as a function of \u\? 

The set A(u) is therefore an “abridged” orbit OrbA u t(F n )(^) that includes 
only those automorphic images of u that have the same length as u does. 

Problem 4.0.3 was motivated by complexity issues for Whitehead’s algo- 
rithm that determines whether or not a given element of a free group of 
finite rank is an automorphic image of another given element. We address 
this problem in Section 4.2.2. 



4.1 Finite Orbits 

In this section, we give a description of finite automorphic orbits. Our 
exposition here follows [286] . 

Let F n be the free group of a finite rank n > 2 with a set X = {x*}, 
1 < i < n, of free generators. Denote by Orb^(w) the orbit of an element u 
of the free group F n under the action of an automorphism p. That is, 
Orb fp(u) = {v £ F n , v = p m (u) for some m £ Z+}. 

The problem that we address in this section is: how many elements can 
a finite orbit like that possibly have if u runs through the whole group F n 
and p runs through the whole group Aut(F n )? The answer is provided by 
the following theorem, in combination with a result of McCool [240] (see 
also [179]): 

Theorem 4.1.1. In the free group F n , there is an orbit Orb^(u) of car- 
dinality k if and only if there is an element of order k in the group 
Aut (F n ). 

Thus, the question above is reduced to another question, that of finding 
out what possible order a torsion element of the group Aut(F n ) can have. 
The latter was answered by McCool [240]; more general results were ob- 
tained later by Khramtsov [179]. We cite the relevant result after the proof 
of Theorem 4.1.1. 

It should be pointed out that the “only if” part of our Theorem 4.1.1 is 
no longer valid if p is an arbitrary endomorphism. The following example 
is based on the idea suggested by C. Sims. 

Example 4.1.2. In the free group F 3 , let p be the endomorphism that 
takes x\ to #2 to #i, and £3 to 1. Let u = £1X2X3; then the car- 

dinality of Orb^(u) is 5, but there is no element of order 5 in the group 
Aut(F 3 ). 
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Proof of Theorem 4.1.1. 

(1) The “only if” part is a combination of an observation due to G. Levitt 
(see [211]) with a result of Bestvina and Handel [51]. Here is the argument. 
Suppose that, for some automorphism p of the group F n , one has p k (g) = g 
and p q (g) ^ g for 0 < q < k. 

Consider the action of p on the subgroup H = Fix(<p fc ) of all elements 
fixed by p k . (This subgroup is clearly invariant under p since p k (p(h)) = 
p(p k (h)) = p(h).) Then p is an automorphism of H. Indeed, p is obviously 
surjective on H since, for any h G H, we have h = p(p k ~ 1 (h)). If p were 
not injective on Ff, then we would have p(h) = 1 for some h £ H, in which 
case h could not be fixed by p k . 

Finally, p clearly has order k as an element of the automorphism group 
Aut (H). Since H has rank at most n by [51], this yields the “only if” part 
of the theorem. 

(2) To prove the “if” part, we need the following definition. A group G 
satisfies the big powers condition if, for any tuple of elements tq, . . . ,u n 
from G with [u^u^+i] ^ 1 (i = 1, . . . , n — 1), there is an integer K such 
that for any integers Mi, . . . , M n > K , the following inequality holds: 

It is known that every free group satisfies the big powers condition [40]. 
Now comes the following: 

Lemma 4.1.3. Let p be a nonidentical automorphism of F n . Then there 
exists an integer K > 1 such that for any Mi, . . . , M n > K the following 
inequality holds: 

Proof. Suppose, by way of contradiction, that for any integer K > 0 there 
are integers Mi (AT), . . . , M n (K) > K such that 

V (*f l(K) . . .x^W) = xf l(K) . ..x^ K \ 

It follows that 

¥>(: Xi ) M 1 W ■ • • <f(x n ) M ^x- M ^ . . . = 1 (4.1) 

for all positive integers K. As we have mentioned above, the free group F n 
satisfies the big powers condition; therefore there are two commuting con- 
secutive factors in (4.1). Since p is an automorphism, the only consecutive 
factors that can possibly commute are p(x n ) Mn ^ and Xn n ^ K \ It follows 
that p(x n ) = x n and (4.1) takes the form 

= 1. 

Upon repeating the argument above, we get p(xi) = Xi for all i = 
1, . . . , n; i.e., p is identical. This contradiction proves the lemma. ■ 
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We now continue with our proof of the “if” part. Given k > 1 and an 
automorphism of order k of the group jF n , we are going to find an element 
u £ F n so that the orbit Orb^(u) has cardinality k. 

If (p is a permutation on the set {rrf 1 , . . . , x then any element of 

the form u = x^ 1 x^ 71 , Mi ± 0, would do. If not, then there is at 

least one free generator, say, xi, such that <p(x i) has length at least 2. Let 

u = x^ 1 x^ n - Then, by Lemma 4.1.3, for some choice of K > 1, for 

any M \ , . . . , M n > K we have (p(u) ^ u. 

Similarly, for any m, 1 < m < fc, we can construct an element u m 
such that (p m (u m ) ^ u m . Every u m , m > 2, is chosen to be of the form 

U m = x^ 1,Tn Xn n,m with min* M^ m > max z M ijm _ i, and ip m (u m ) ± u m 

(the latter is possible by Lemma 4.1.3). 

Obviously, with this choice of M^j we will also have <p*(uj) ^ Uj for any 
j < m. Therefore, for u = Uk, the orbit Orb^(u) will have cardinality k. 



We note that possible values of the order of a torsion element of the 
group Aut(F n ) are described, according to [240] and [179], as follows. 

Pick a positive integer k = p* 1 p" s , where pi, . . . ,p s are different 

primes. There is an element of order k in the group Aut(F n ) if and only if 
^2i-i(pT ~pT~ 1 ) — n ' F° r example, if fc = 15 = 3*5, then the sum above 
becomes (3 — 1) + (5 — 1) = 6. Therefore, there is an automorphism of order 
15 in the group Aut(F n ) for n > 6 but not for n < 5. 

We also note that Levitt and Nicolas [210] proved that the maximum 
order (call it H(n)) of a torsion element of Aut(F n ) is the same as that of 
a torsion element of GL n (Z), with the exception of n = 2, 6, and 12. They 
also established the asymptotics of this function by showing log H (n) ~ 
yjn • logn. 



4.2 Abridged Orbits 

In this section, we study “abridged” automorphic orbits 

A(u) = {v e F r \ \v\ = |u|, f(v) = u for some / G Aut(F r )}. 

One of the most interesting special cases is where u = x i. In other words, 
A(x i) is the set of primitive elements of a given length in F r . 

4-2.1 Counting primitive elements 

Let P(r, n) be the number of primitive elements of length n in F r . For r = 2, 
the number P( 2, n) grows like (>/3) n ; this follows from [63, Proposition 4.1], 
as observed by Rivin [315]. 

We also note that the precise number of cyclically reduced primitive 
elements of length n in F 2 was given in [286] (see Proposition 4.2.4 below); 
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this number has linear growth as a function of n. This case however is 
exceptional, and it is easy to see that if r > 3, then the growth of the 
number of primitive elements (cyclically reduced or not) is exponential, 
say, A n . The problem is therefore to determine A (as a function of r). 

It was shown in [63] that A < fi(r ) > 2r — 2, where /jb(r) — > (2 r — 2) 
as r -» oo. In [58], it was shown that A < 2r — 2. In both papers, crucial 
for the counting arguments was the fact that the Whitehead graph of any 
primitive element of length > 2 has either an isolated edge or a cut vertex, 
i.e., a vertex that, having been removed from the graph together with all 
incident edges, increases the number of connected components of the graph. 

Here we again exploit this fact to sharpen the estimate for A: 

Theorem 4.2.1 ([358]). Ifr> 3, then , for some co nstants c\, C 2 , one has 
ci-n-(2r-3) n < P(r,n) < C 2 *A n , where A = r-l + ^/(r — l) 2 — 1 < 2r-2. 

It seems that this estimate exhausts the potential of the “cut vertex” 
approach, and further improvements would require brand new ideas. As 
for the ultimate result, we make the following 

Conjecture 4.2.2. If r > 3, then P(r, n) = 0(n • (2r — 3) n ). 

We note that n • (2r — 3) n is (up to a factor that does not depend on n) 
the number of “obvious” primitive elements of length n having the form 
u • xf 1 • v, where u, v are arbitrary elements that do not depend on X{. 
Thus, our conjecture says that, for r > 3, “most” primitive elements are of 
that form. 

Our proof here uses a refined analysis of the Whitehead graph of a prim- 
itive element and a new ingredient, namely, systems of linear recurrence 
relations, which is described below. We believe this technique can be used 
for other counting problems in (free) groups as well. 

The Whitehead graph Wh(w) of a word u £ F r is obtained as follows. The 
vertices of this graph correspond to the elements of the free generating set X 
and their inverses. For each occurrence of a subword X{Xj in the word u , 
there is an edge in Wh(u) that connects the vertex Xi to the vertex x~ l \ 
if u has a subword XiX ~ x , then there is an edge connecting Xi to Xj , etc. 
There is one more edge (the external edge) included in the definition of the 
Whitehead graph: this is the edge that connects the vertex corresponding 
to the last letter of u to the vertex corresponding to the inverse of the first 
letter. 

It was observed by Whitehead himself that the Whitehead graph of any 
primitive element of length > 2 has either an isolated edge or a cut vertex 
(i.e., a vertex that, having been removed from the graph together with 
all incident edges, increases the number of connected components of the 
graph). 

Both [42] and [63] exploited this fact to get upper bounds for P(r, n). 
Here we refine these upper bounds based on the following simple 
observation (see, e.g., [402, Exercise 5.7]): 
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Suppose a simple graph T has the property that any two vertices can be 
included in a simple circuit Then T does not have a cut vertex. 

Our strategy therefore will be to produce an upper bound for the number 
of elements u G F r whose Whitehead graph (or, rather, an associated simple 
graph) does not have the property alluded to in the previous observation. 

Our counting technique is best visualized by introducing the Whitehead 
tree WT(w) associated to the graph Wh(u). The Whitehead tree WT(w) is 
an infinite rooted labeled tree; the root is labeled 1, and a parent vertex 
at a level k labeled Xi is connected to a child vertex at the level ( k + 1) 
labeled Xj if and only if there is an edge from the vertex Xi to the vertex xj 1 
in the graph Wh(u) (or, equivalently, if u has a subword XiXj). 

Now the problem of finding the growth function of the tree WT(tt) (i.e., 
counting elements at a level n as a function of n) can be translated into 
a system of (linear) recurrence relations as follows. Let j n (resp. j n ) be 
the number of vertices labeled Xj (resp. xj 1 ) at the level n of the tree 
WT (u). Then, based on the Whitehead graph Wh(u), it is easy to write 
down recurrence relations for all j n+ i and j n+ i in terms of appropriate 
i n and i n . All these recurrence relations are obviously linear, so one can 
employ a well studied theory of systems of linear recurrence relations to 
solve for all j n and j n . 

Solving a system like that basically amounts to finding eigenvalues of the 
matrix of this system, usually called the transition matrix. (Note that all 
entries of this matrix are nonnegative.) In fact, since we are only interested 
in the growth of j n + j n , we only need the maximum real positive 
eigenvalue (the Perron-Frobenius eigenvalue) of the transition matrix; this 
will determine the growth type of the tree WT (u). 

Of course, a system like that is, in general, too big; it has 2 r equations. 
In most practical situations, however, the set of vertices of the Whitehead 
graph Wh(u) can be split into a union of a few (large) sets, and then one 
can write recurrence relations (still linear) for the numbers of elements in 
these sets, thus getting a reasonable number of equations. 

Finally, we mention that some sets of elements of F r can be interpreted 
as Markov processes whose transition matrices arise in a way similar to 
what we described above. It is conceivable that sets like that are precisely 
graphic sets in the sense of [63]. 



Proof of Theorem 4.2.1. Let u £ F r , and let Wh(u) be the und erlying 
simple graph of the Whitehead graph Wh(u) (in particular, Wh(u) has no 
multiple edges). 

There are two principal cases to consider; 

Case 1 . There is a vertex of degree 1 in Wh(n). 

Case 2. The degree of any vertex of the graph Wh (u) is > 2. 

In Case 1, suppose that, say, the vertex of Wh('u) corresponding 
to the generator x\ has degree 1. That means u has the form u = 
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u(xiXj, £2, • • • 5 x r ) for some j. Then the vertex corresponding to xj 1 may 
have any degree < 2r — 1, whereas other vertices have degre es < 2 r — 2. 

Thus, t he greatest possible number of edges in the graph Wh(u) occurs 
if Wh('u) has one vertex of degree 1, one vertex of degree 2r — 1, and 
2r — 2 vertices of degree 2r — 2. To produce the corresponding transition 
matrix, we let a n be the number of vertices labeled x\ in the nth level 
of the Whitehead tree WT(w), b n the number of vertices labeled xj 1 , c n 
the number of vertices labeled Xj, and r n the number of vertices labeled 
by other letters. Then we have the following system of linear recurrence 
relations: 



^n+l — “1“ Cn T n 

< bn+i — b n + r n ^ ^ 

£n+l ^ 4 ~ C n “b T n 

Jn+i = (b n + Cn) • (2 r - 3) + r n • (2 r - 4) 

We note that the actual value of r n + 1 is smaller than (b n +c n ) • (2r — 3) + 
r n • (2r — 4) because, say, some vertices labeled Xj may be connected to 
vertices labeled Xj, and not to vertices labeled by “other” letters. However, 
since we are looking for an upper bound, we are allowed to replace the 
actual value of r n _|_i by a greater one. 

The transition matrix of this system is 

(0 1 1 1 \ 

0101 
10 1 1 

^0 2r — 3 2r — 3 2r — 4y 
The characteristic polynomial of this matrix is 

A 4 - (2 r - 2)A 3 + (2 r - 2)A - 1 = (A 2 - 1)(A 2 - (2 r - 2)A + 1), 
and the Perron-Frobenius eigenvalue is 

Amax = r- 1 + \/{r - l) 2 - 1. 

Thus, the number of elements of length n in Case 1 is 0((r — 1 + 

V(r-l) 2 -!)")• 

In Case 2, we start with a simple 

Lemma 4.2.3. Suppose the degree of any vertex of a simple graph T with 
2 r vertices is >2. If T has either at least two vertices of degree > 2r — 2 
or at least one vertex of degree 2r — 1, then any two vertices of T can be 
included in a simple circuit. 

Proof. Let v\ and v<i be two vertices of degree > 2r — 2. If they are 
not adjacent, then they both must be adjacent to all other vertices of the 
graph T. Then, given any two vertices w\ and w 2 of T, we can include them 
in the simple circuit w\ — )> v 2 w 2 — > v\ — > w \ . 
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If vi and v 2 are adjacent, then there might be a vertex w which is not 
adjacent to either vi or v 2 (or both). Suppose and w 2 are two vertices 
of T. If they both are different from w, then they obviously can be included 
in a simple circuit. Now suppose, say, w\ = w. Since the degree of any 
vertex of T is > 2, w must be adjacent to a vertex ^ w 2 . Then we get 
a simple circuit w — ^ — > v 2 — >* w 2 — >* v\ — » w. 

The case where T has a vertex of degree 2r — 1 is similar. ■ 

From Lemma 4.2.3, we see that if u E F r is a primitive element, then at 
most one vertex of the graph Wh(u) can have degree > 2r — 2, and there 
are no vertices of degree 2r — 1. If there are no vertice s of deg ree 2r — 2, 
then the number of elements of length n with the graph Wh(it) like that is 
obviously bounded by (2r — 3) n , so there is nothi ng to pr ove. 

Thus, the greatest possible number of edges in Wh(u) occurs if Wh(u) 
has one vertex of degree 2r — 2, and 2r — 1 vertices of degree 2r — 3. We 
no te in p assing that, in fact, this cannot happen because, i f all ve rtices 
of Wh(u) have degrees > 2r — 3, then, since 2r — 3 > y , Wh(u) must 
be Hamiltonian by a classical result of Ore (see, e.g., [402, Theorem 7.1, 
Corollary 7.2]). A Hamiltonian graph cannot have a cut vertex. However, 
since we are looking for an upper bound, this is legitimate. 

Let now the vertex of degree 2 r — 2 have label x\. It is clear that, for any 
element u whose graph Wh(u) has properties mentioned above, at every 
level of the Whitehead tree WT (u) one has at least 2r — 4 times as many 
vertices labeled Xi, i 7^ 1, as one does vertices labeled x\. We therefore have 
the following inequalities for the numbers r n of vertices at level n : 

r„ +1 <^-r n . ( 2 r- 2) + ^|-r n - ( 2 r- 3) = r n .( 2 r- 3 +^ 3 ). 

Thus, r n < c • (2r — 3 4- 27^3) n f° r some constant c in this case. Since 
2r — 3 + 2^33 < r — 1 + y/(r — l) 2 — 1, this completes the proof of the 
theorem. ■ 

For the group F 2 , the number of cyclically reduced primitive elements 
of length m > 1 can be computed precisely; this number happens to have 
a rather slow growth (as a function of m). If, however, we drop the condition 
of being cyclically reduced, the situation changes, and the growth becomes 
exponential. 

Proposition 4.2.4 ([286]). The number of primitive elements of length m 
in the group F 2 ( and therefore, in any group F r , r > 2) is: 

(a) bigger than • (\/3) m if m is odd; 

(b) bigger than | • (\/3) m if m is even; 

(c) The number of cyclically reduced primitive elements of length m> 1 
in the group F 2 is 4m • 4>(m) ? where $(m) is the Euler function of m 
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(i.e., the number of positive integers < m relatively prime to m). 
( Clearly , $(m) < m.) 

Informally speaking, “most” primitive elements in F 2 are conjugates of 
primitive elements of smaller length. Rivin [315] observed that the number 
P(2,ra) of all primitive elements of length m in F 2 actually grows like 
(\/3) m ; i.e., the lower bounds given in Proposition 4.2.4 are rather tight in 
the case of F 2 . 

Proof of Proposition 4.2.4 Let x and y be generators of F 2 . 

(a) Suppose m is odd. Then any conjugate of x ±l , as well as of y ±l , by 
an element of length k = (m — l)/2, is a primitive element of length m (as- 
suming there are no cancellations in the middle). The number of elements 
like that in the group F 2 is 2 • S^ -1 , whence the result. 

(b) If m is even, then counting conjugates of x ±x y and xy ±x by elements 
of length (m — 2)/2 yields the result. 

(c) The result of this part will follow from a well-known fact about prim- 
itive elements of F 2 (see [71] or [296]): for any pair {&,/} of integers with 
( k,l ) = 1, there is exactly one cyclically reduced primitive element of F 2 
whose exponent sum on x is k and the exponent sum on y is l. 

Thus, the number of cyclically reduced primitive elements of F 2 of length 
m is 8 times the number of pairs {fc, 1} of positive integers with (k, l) = 1, 
k < Z, k + l = m. The latter number is obviously equal to -$(m), where 
4>(ra) is the number of positive integers < m relatively prime to m. ■ 

4-2.2 Complexity of Whitehead’s algorithm 

In this section, we study the complexity of Whitehead’s algorithm that 
determines whether or not a given element of a free group of finite rank is 
an automorphic image of another given element. 

It is known that the first part of this algorithm (reducing a given free 
word to a free word of minimum possible length by “elementary” White- 
head automorphisms) is pretty fast (of quadratic time with respect to the 
length of the word). On the other hand, the second part of the algorithm 
(applied to two words of the same minimum length) was always considered 
very slow. In fact, the procedure outlined in the original paper by White- 
head [400] suggested this part of the algorithm to be of superexponential 
time with respect to the length of the words. Indeed, given a word u, the 
procedure calls for constructing a graph whose vertices correspond to all 
words of length \u\. This means that the number of vertices is an exponen- 
tial function of \u\. After that, for every vertex of the graph, one constructs 
edges incident to this vertex as follows: an edge connects this particular 
vertex to another vertex if and only if there is an elementary Whitehead 
automorphism that takes one of the corresponding words to the other. Fi- 
nally, to find out if there is an automorphism that takes the word u to 
another given word v of the same length, one has to check all the paths in 
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the graph that start at the vertex that corresponds to u and see if some 
of them lead to the vertex that corresponds to u. The number of paths in 
a graph is, in general, an exponential function of the number of vertices. 
Therefore this algorithm is, in general, of superexponential complexity with 
respect to the length of the word u. 

It is possible, however, to skip some steps in this algorithm and get the 
following: 

Proposition 4.2.5. Let N be the number of automorphic images ofu G F n 
that have the same length as u does. Then, given an element v of length \u\, 
one can decide in quadratic time with respect to N whether or not v is an 
automorphic image ofu. 

Proof. We are going to use the backtracking method, which is a well-known 
procedure in graph theory for searching a tree. 

Starting with the vertex that corresponds to u = uq, we are building 
a tree as follows. (We use the same notation for words and corresponding 
vertices when there is no ambiguity.) 

(1) Apply an arbitrary elementary Whitehead automorphism to uo; if 
a new word u\ of the same length is obtained, plot the corresponding vertex 
and connect it to uq. If not, then apply another elementary Whitehead 
automorphism until you get a new word u\ of the same length. (Note that 
the total number of those automorphisms C = C(n) is finite and depends 
on the rank n of the group F n only.) 

(2) Continue the same process. That is, suppose we have obtained 
a word Ui, i > 0, at the previous step. This time “a new word” would 
mean a word different from all the words obtained at previous steps. 

If none of the elementary Whitehead automorphisms produces a new 
word, then do “backtracking” (i.e., return to the word obtained at the 
immediately preceding step, and repeat the same process). 

In the end (i.e., when no new word can be obtained from any of the “old” 
words), we shall obviously have a spanning tree of the graph described 
before the statement of Proposition 4.2.5. It will therefore have N vertices 
and N - 1 edges. Furthermore, in the course of constructing this tree, we 
did not visit any of the vertices more than N times because to obtain any 
new vertex we had to visit other vertices no more than once. 

Thus, the time we need to construct this tree is no more than C • iV 2 , 
where C is the constant mentioned above. Once the tree is constructed, it 
will take just N more steps to find out if the vertex corresponding to the 
word v is among the vertices; or, we can perform the check every time we 
get a new vertex because once we get v we can stop. ■ 

Furthermore, in the case where the free group has rank 2, we have 

Theorem 4.2.6 ([286]). Let u<EF 2 be a word whose length is irreducible 
by any automorphism of F 2 {in particular, u is cyclically reduced). Then 
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the number of automorphic images ofu that have the same length as u does 
is bounded by a polynomial function of \u\. 

In fact, experimental data suggest that the number in the statement of 
Theorem 4.2.6 has the (exact!) bound of 8m 2 — 40m for m > 9, where 
m = \u\ (and, therefore, 8m — 40 if one counts cyclic words). Recently, 
D. Lee [207] was able to prove this. 

Theorem 4.2.6 has the following corollary: 

Corollary 4.2.7. In the group F 2 , Whitehead 7 s algorithm terminates in 
polynomial time with respect to the maximum length of the two words in 
question. 

By the aforementioned result of D. Lee, Whitehead’s algorithm in F 2 
actually terminates in quadratic time with respect to the maximum length 
of the two words. 

We also mention here an interesting “statistical” result from [172] even 
though it is outside of the scope of this book: for “most” inputs (in a precise 
sense), Whitehead’s algorithm in F n , n > 2, terminates in linear time with 
respect to the maximum length of the inputs. 

We do not know whether or not Theorem 4.2.6 and, therefore, Corol- 
lary 4.2.7 hold for free groups of bigger ranks. However, experimental data 
kindly provided by C. Sims allowed Myasnikov and Shpilrain [286] to make 
the following conjecture: 

Conjecture 4.2.8 ([286]). In the free group F n , the cardinality of A(u) 
is bounded by a polynomial of degree 2n — 2 in |u|, provided the length of 
u is irreducible by any automorphism of F n . 

A most amazing thing is that, according to the experimental data men- 
tioned above, the maximum cardinality of A{u) that can actually occur 
under the irreducibility assumption in Conjecture 4.2.8 appears to be pre- 
cisely a polynomial of degree 2n — 2 in m = \u\ for sufficiently large m. 
For n — 2, this polynomial, as we have already mentioned, is 8m 2 — 40m 
if m > 9. For n = 3, the polynomial is 48m 4 — 480m 3 + 1104m 2 — 672m 
if m > 11. A particular element u 6 F3 of length m whose orbit A(u) has 
the cardinality given by the latter polynomial is, according to the same 
experimental data, u = x\x 2 X\x^ x x\x\x\, where k = m — 8. 

D. Lee [207] was able to prove, under some mild restrictions on u, that 
the cardinality of A(u) is bounded by a polynomial of degree n(3n — 5)/2 
in \u\. 

We also note that, in the case where the free group has rank 2 (but 
not in the general case), the condition on \u\ to be irreducible by any 
automorphism can be relaxed to u just being cyclically reduced. 

Proof of Theorem 4.2.6. Throughout the proof, we shall call “length- 
preserving” those automorphisms of F 2 that are permutations on the set 
{x, x _1 , y, y' 1 }. There are eight of them, so whenever we count the number 
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of automorphic images of a particular element “up to a length-preserving 
automorphism”, it means the upper bound for such a number should be 
multiplied by 8. 

Let M = \u\. Let k be the sum of exponents on x in the word u, and 
let l be the sum of exponents on y. Upon applying a length-preserving 
automorphism if necessary, we may assume that fc, l > 0. First, we are going 
to establish the result of Theorem 4.2.6 for u £ [F 2 , F 2 ], so we assume that 
fc,Z are not both 0. In this case, the result will follow from the following 
observations. 

(1) For a word of length M, there are ]U^ 0 (i + 1) = \(M + 1)(M + 2) 
possible pairs (fc, l ) with A;, / > 0; k + l < M. 

(2) It is well-known (see, e.g., [231]) that the group Aut(F 2 ) is gener- 
ated by inner automorphisms, by three length-preserving automorphisms 
tt: x -> y, y x\ cr x : x -» x~ l , y -» y\ a y \ x -> x, y -¥ y~ l , and by the 
following two: a : x xy, y — )> y, and /3 : x — > x, y yx. The subgroup H 
of Aut(F 2 ) generated by a and (3 can be mapped onto SL 2 (Z). Under this 
epimorphism, a and j3 correspond to the matrices (\i) and ( J \ ), respec- 
tively. The kernel of this epimorphism is generated (as a normal subgroup) 
by the inner automorphism induced by the element [x,y\\ in particular, 
every automorphism in the kernel is inner. 

Furthermore, relations between generators of Aut(F^) given in [231, Sec- 
tion 3.5, Theorem Nl] show that, in any product of automorphisms a^ 1 , 
Z^ 1 , 7 r, <j x , and o y , automorphisms a ±x and (3 ±l can be collected on the 
right. This, together with the fact that the subgroup of inner automor- 
phisms of F 2 is normal in Aut(F 2 ), implies that applying an automorphism 
of F 2 amounts to first applying an automorphism from the subgroup H gen- 
erated by a and /?, then a length-preserving automorphism, and, finally, an 
inner automorphism. 

Therefore, to bound the number of cyclically reduced automorphic im- 
ages of u with the same nonzero vector (fc, l) of exponent sums, it is 
sufficient to bound the number of matrices from SL 2 (Z) that fix the vector 
(fc,Z) acted upon by right multiplication and then multiply this number 
by M (the number of cyclic permutations of a word of length M). Further- 
more, up to a length-preserving automorphism, every automorphism from 
the group H corresponds to a matrix from SL 2 (Z) whose elements in the 
first row are of different signs, say, the element in the upper left corner 
is nonnegative and the element in the upper right corner is nonpositive. 
(Elements in the first row correspond to the image of x.) 

(3) Thus, what is left to do now is to count the number of matrices in 
SL 2 (Z) whose elements in the first row are of different signs that fix a given 
nonzero vector (fc, l) with k,l > 0. The computation here is straightforward. 
Let A = (all all ) a matrix from SL 2 (Z) with an > 0, ai 2 < 0, which 




4.2. Abridged Orbits 57 



fixes a vector (fc, Z). Then we have the following system of equations in a^: 

k • an ^ * &21 = k , 

A: • ai2 + l * a22 — Z, 

«na22 — «i2a2i = 1. 

Suppose first that both fc, l ^ 0. Then, from the first equation, we get 
&2i = j — f • an, and from the second equation a22 = 1 — y • ai2- Plug 
this into the third equation and simplify: Z • an — k • ai2 = Z. Since fc, l > 0, 
an > 0, ai2 < 0, this gives either ai2 = 0, an = 1, or an = 0, ayi — 

In the former case, we get a22 = 1, a2i = 0. In the latter case, 021=7, 
an = 0, a22 = 2. 

Now suppose, say, k — 0. Then a2i = 0, a 2 2 = 1, an — 1, whereas ai 2 
can be arbitrary. However, we can show that, should the automorphism 
corresponding to the matrix A preserve the length of a, the absolute value 
of ai2 cannot be greater than 2 \u\. Indeed, let K = a\ 2 \ then the auto- 
morphism corresponding to the matrix A is a K (i.e. , it takes x to xy K , 
y to y). Suppose K > 2|u|; we may assume that u has at least one occur- 
rence of x. Then a K {u) has a subword xy K (before cancellation). Since we 
have assumed that a K (u) has the same length as u does, more than half 
of y K should cancel out. This implies that, in the word u itself, there is 
a subword y~~ N with N > ^2^- This is a contradiction since K > 2 \u\. 

Thus, in any of the cases considered, we have no more than 2 \u\ different 
matrices from SL,2(Z) that fix a given nonzero vector (fc, /). 

Summarizing the observations (1), (2), and (3), we see that the number 
of cyclically reduced automorphic images of u of length M = \u\ is no more 
than c • M 4 for some constant c independent of u. This completes the proof 
in the case where u ^ [F2, Fq\. 

Now let u € [^2,^2]- In this case, we are going to use induction on the 
length of u. To make the induction work, we are going to prove the following 
somewhat stronger claim. 

Proposition 4.2.9. Let u E [F2, F2] be cyclically reduced. For any positive 
integer K , the number of elements v e F 2 such that v = <p(u) for some 
<p G Aut(F2) and |u| = \u\ + K is less than c • 3 K • (|u| + K ) 4 for some 
constant c independent of u and K. 

Proof. The basis of induction u = [x,y\ is almost obvious. This element 
is fixed by any automorphism from H (recall that H is the subgroup of 
Aut(F 2 ) generated by two automorphisms, a : x — > xy , y — »■ y, and f 3 : x — >• 
x, y -» yx), and therefore, to count the number of elements v G F 2 such 
that v = (p(u) for some G Aut(i<2) and \v\ = \u\ + K , we just have to 
count (up to a length-preserving automorphism) the number of conjugates 
of u of length up to |u| + K. This latter number is no bigger than the 
number of different elements of length [K/ 2] in the group F 2 (i.e., it equals 
3 ^/ 2 ] ). 
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For the induction step, we first assume that u has a subword of the 
form [x ±1 ,2/ ±1 ]. Then, upon applying a length-preserving automorphism 
if necessary, we may assume that u has a subword [x,y]. Then a cyclic 
permutation of u has the form [x,y]w, with no cancellation between [x,y] 
and w . Thus, by the remarks in the beginning of the proof of Theorem 
4.2.6, we may assume that u= [x,y]w. Let p be an arbitrary automorphism 
from H. Recall that every automorphism in H fixes [x,y]. 

Assume first that w is cyclically reduced. We have two possibilities: 

(1) [x, y] is entirely cancelled out by p(w). Then, since <p(u) = [x, y]p(w), 
we see that, if \<p(u)\ = \u\ 4- K , we must have \<p(w)\ = \w\ + K + 4. By 
the inductive assumption, the number of automorphic images of w with 
this property is no more than c • 3^ +4 • | w + K + 4| 4 for some constant c 
independent of w and K. A similar result for u now follows. 

(2) Only part of [x, y] cancels out (this includes the case where nothing 
cancels out). Then, since p(u) = [x,y]<p(w) and since an element of the 
commutator subgroup must have an even length, we see that, if \<p(u)\ = 
\u\ + K , then either \p{w)\ = \w\ + K + 2 or \ip(w)\ = \w\ + K. By the 
inductive assumption, the number of automorphic images of w with this 
property is no more than c • 3 K+2 • | w -F K + 2 1 ' 4 (respectively, c • 3 K • | w + K \ 4 ) 
for some constant c independent of w and K. A similar result for u now 
follows. 

If w is not cyclically reduced (i.e., if u = [x, y\gw'g~ 1 ), then we consider 
a cyclic permutation of u : u' = <? -1 [x, y]gw f = [ x 9 , y 9 ]u/, where we can 

assume w ' to be cyclically reduced. Now we apply essentially the same ar- 
gument to v! as we have just applied to u, upon replacing the subgroup H 
of automorphisms by the left coset i g H , where i g is the inner automorphism 
induced by the element g. (Applying an automorphism from % g H is equiva- 
lent to first applying conjugation by g and then applying an automorphism 
from H.) 

Since the group of inner automorphisms is normal in Aut(F2), obser- 
vation (2) in the beginning of the proof of Theorem 4.2.6 remains valid 
upon replacing H by i g H. That is, every automorphism from Aut(F 2 ) is 
a product of an automorphism from the coset i g H and an inner automor- 
phism. Since every automorphism from i g H fixes the element [x 9 ,y 9 ], 

the same argument as above completes the proof in this case. 

Suppose now that u does not have a subword of the form [x ±1 ,y ±1 ] 
but does have a subword of the form x ±1 y ±1 x =Fl . Then, upon applying a 
length-preserving automorphism if necessary, we may assume that u has 
a subword xyx~ l . Thus, a cyclic permutation of u has the form xyx~ x w , 
with no cancellation. Then we can write u as u = [x,y\yw. Note that 
the word yw has smaller length than u does, and we can assume that yw 
is cyclically reduced, for if it was not, w would end with y -1 , and then 
a cyclic permutation of u would be of the form y~ x xyx~ x w' — [?/ _1 ,x]u/ , 
and therefore this case would be reduced to the previous one. 
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Thus, we can apply the inductive assumption to this word w , and the 
same argument as above will work in this case as well. 

Finally, suppose that u does not have a subword of the form x^y^x^ 1 . 
Then u must have a subword of the form x ±1 y k x Tl for some k ^ 0, ±1. We 
can assume, upon applying a length-preserving automorphism and a cyclic 
permutation if necessary, that u = xy k x~ 1 w 1 k > 1. Then we can write 
u = [x,y]yxy k ~ 1 x~ 1 w. Now the word yxy k ~ 1 x~ 1 w has the same length as 
u does, but it has the subword xy k ~ l x~ l . Also, we can assume that yw 
is cyclically reduced, for if it was not, w would end with y -1 , and then 
a cyclic permutation of u would be of the form y~ 1 xy k x~ 1 w' (i.e., it would 
begin with y~ x xy ), and therefore this case would be reduced to one of those 
previously considered. An obvious inductive argument now completes the 
proof. ■ 



4.3 Endomorphisms 

that Preserve Automorphic Orbits 

In this section, we are going to give a solution to Problem 4.0.2. Our 
exposition here follows [206] . 

As usual, let F n be the free group of a finite rank n on the set X = 
{aq, . . . , x n } of free generators. 

Theorem 4.3.1. Suppose p is an endomorphism of F n such that 
<£>OrbAut F n (W) C OrbAut F n (W) for some W G F n , W ^ 1. Then (p is 
an automorphism of F n . 

Recall that a sequence it = (iq, tq, . . . ) of words of F n is called N -reduced 
(Nielsen reduced) if every Ui G it is reduced and, for any triple 
of the form uf 1 , where Ui G it, the following three conditions hold (see 
Section 1 . 1 ): 

• (Nl) v 1 ^ 1; 

• (N2) viv 2 7 ^ 1 implies |viv 2 | > |vi|, \v 2 \; 

• (N3) viv 2 7 ^ 1 and V 2 V 3 ^ 1 implies \v\V 2 Vz\ > |tq| - \v 2 \ + \vz\. 

It is well-known that if it = ( 14 , u 2l . . . ) is iV-reduced, then the subgroup 
(it) of F n generated by it is free with the basis it. Also, every finite sequence 
of reduced words of F n can be carried by Nielsen transformations (see 
Section 1 . 1 ) to some iV-reduced sequence. 

Recall also that a Whitehead automorphism u of F n is an automorphism 
of one of the following two types (see Section 1 . 2 ): 

• (Wl) cu permutes elements in 
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• (W2) uj is defined by a set S C X ±l and a letter a £ X ±x with 
a £ S and a~ l £ S in such a way that if x £ X ±x then (a) u(x) = x 
provided x = a ±1 ; (b) u(x) = xa provided x ^ a, x £ <S, and x -1 ^ <S; 
(c) a;(x) = a~ x xa provided both x,x -1 £ <S; (d) u;(x) = x provided 
both x, x _1 ^ S. 

If a; is of the type (W2), then we write uj = cj(S,a). We also need 
the standard Whitehead graph $(Y), which is constructed as follows (see, 
e.g., [227]). Take X ±l as the vertex set, and connect two vertices a, b £ X ±x 
by a nonoriented edge if there is a subword ab~ l or ba~ l of (noncyclic) Y. 
We let C(a, Y) denote the connected component of $(Y) containing a 
vertex a £ $(Y). 

As usual, by writing X = Y we mean the graphical equality (letter-by- 
letter) of words X,Y £ F n (whereas X — Y means the equality of AT, Y 
in F n ). By X we denote a reduced word equal in F n to X and by \X\ the 
length of a word X of F n with respect to the basis X. 

Proof of Theorem 4.3.1. Without loss of generality, we may assume 
that W has the minimum length over all words in OrbAut F n (W). Then 
the rank of W is defined to be the number of elements of X occurring 
in W. We consider two cases corresponding to whether W has rank 1 or 
not. 

Case 1. W has rank 1. 

If W is a primitive element of F n , then there is nothing to prove due 
to the result of [205]. Also, if W is a proper power of a primitive element 
of F n , then the assertion follows immediately by combining the result of 
[205] and the fact that extraction of roots in a free group is unique. 

Case 2. W has rank r with 1 < r < n. 

Upon permuting elements of X, we can assume that only the elements 
xi, X 2 , . . . , x r of X occur in W. We may further assume, by interchanging 
Xi and x~ x and then permuting the elements xi,X2,...,x r if necessary, 
that 

x\ has positive exponent sum in W 

unless every X{ has zero exponent sum in W. (4.3) 

Now let u be a primitive element of F n . Due to the result of [205], it is 
enough to prove that ip(u) is also primitive. By definition of a primitive 
element, there is a £ Aut F n such that a(xi) = u. Note that ipoa has the 
same property as </? does; that is, (ip o a) OrbAut F n {W) C OrbAut F n (W). 
Hence, upon replacing <poa by ip, it suffices to prove that ip(x\) is primitive. 

Let F r , F n+ 1 be the free groups on the sets X r = {xi, X 2 , . . . , x r }, 
£ n+ i = {xi, X 2 , . . . , x n _|_i}, respectively. Obviously, F r is a free factor of F n 
of rank r. Now we want to prove: 

Claim 1. The restriction (p\p r of p to F r is a monomorphism. 
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Proof. Suppose, by way of contradiction, that p\p r is not a monomor- 
phism. Consider the finite sequence (p(xi), ( / ? ( x 2 ), • • • , <p(av))- This se- 
quence can be brought to an iV-reduced one by Nielsen transformations. 
This fact implies the existence of 5 G Aut F r such that (p o 5)(xi) = • • • = 
(p o S)(xk ) = 1 and that the sequence ((p o 5)(xk+ 1 ), . . . , (p o S)(x r )) is 
iV-reduced. (Note that k > 1 since p\p r is not a monomorphism.) For the 
sake of simplicity of notation, we put 

Ui = (ip o 5)(xi) for i = k + 1, . . . ,r. 

Define e G Aut F r as 

e(xi) = xi[sk + i4+2 • ■ - 4 - 144-1 • --4+24+1] and 

e(xi) = Xi for i = 2, . . . , r, 

where l is a sufficiently large integer. Now consider (p o 5 o e)(W). Since 
(5 o e)(W') belongs to Orb Aut F n (W0, (p ° 6 o ^)(TF) is in Orb Aut F n (W0 by 
the hypothesis of the theorem. If k = r — 1, then by (4.3) we have either 
(p o 5 o e){W) = 1 or (p o S o s)(W) = with i' being sufficiently large 
(so l' > | WF|). The former case contradicts (po8o s)(IT) G Orb Aut F n (W). 
So does the latter case because, for simple words u and v in F n (i.e. , u and 
v are nonempty, cyclically reduced, and not proper powers), the equality 
u m = v m with m, m! > 0 implies that u — v and m — m f . Hence, k has to 
be less than r — 1. 

Let (3: F n — »• F n +i be a homomorphism given by 

(3(x i) = x n +i and /3(x<) = (p o S)(xi) for i = 2, . . . , n. 

Since (<£>o£)(IF) G Orb Aut f u (W) by the hypothesis of the theorem, clearly 
(p o 5) (W) ^ 1. This implies that we may assume, by permuting the 
elements x\, x 2 , . . . , x r if necessary, that 

at least one x n +\ remains uncancelled in /3(W). (4.4) 

It follows from (p o 5) (#i) = • • • = (p ° 5)(xk) = 1 that 

(poSo £ )(W) = Z 0 K tx Z\ K t2 Z 2 . . . K**Z P , 

where K = U ( k+1 U e k+2 . . . • • • ^+ 2 ^+ 1 * f ^ Z \ M- 1 < P < 

| W I, Zi is a product of elements of {U^ v . . . , C/^ 1 }, and only Z 0 and Z p 
may be trivial. Since (p o S o e)(W) G Orb Aut F n (lT), there is ( G Aut F n 
such that (( opo 5 o e)(W) = W. For such ( G Aut F n , we have 

w = ((opo5o £ )(w) = azo) c w* 2 c(^) . . . 

(4.5) 

Let us write 



C(Ui) = YiX^Y- 1 for i = k + 1, . . . ,r, 
where Si G Z + and Xi is a simple word in F n . 
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Suppose that the product on the right-hand side of (4.5) involves a can- 
cellation between Xf Si£ and xf Sj£ such that a part of Xf Si£ of length 

greater than \X{\ + \Xj \ cancels with a part of X± Sj£ of length greater than 
| Xi | + 1 Xj | . We first assume that such a cancellation occurs in a subproduct 
of the form 

X- s ' l Y-\(Z)Y 3 Xf, 

where Z is a product of elements of {Uj^; v . . . , U^ 1 }. It is not hard to prove 
that if u, v are simple words in F n and w is a subword of both u m and v 171 ' 
with m,m' > 0 such that \w\ > \u\ + |v|, then u is a cyclic permutation 
of v. This implies that Xi is a cyclic permutation of Xj. So, if the part of 
Xj j£ that cancels with a part of X~ Si£ begins with T, where \T\ = \Xj\, 
then we can write Xi = ATA~ X and Xj = BTB~ l for some A,Be F n . 
From the fact that no nontrivial cyclic permutation of a simple word u can 
be u itself, we must have 

A^Xrnyr^Z) Yj Xf B = 1 
for some m, m' € Z+ U {0}. Thus, 

1 = A- 1 X" m y- 1 C(Z) Yj xfB = T~ m A~ 1 y- 1 C(Z) Yj B T m> , 
and hence 

c (Z) = Yi AT m ~ m ' B~ 1 Y~ 1 . (4.6) 

Consider ((U i Sj ) £(Z) £([/?*). We can observe that 

au; Si )az) c(^) = ^ x-^v-'az) y 3 xpY- 1 

= YiAT~ s ' s i A-'Y-^^Yj BT SiS > B-'Yf 1 
= YiAT m - m ' B-'Yf 1 by (4.6) 

= C (Z) again by (4.6). 

It then follows from ( G Aut F n that 

Ui^ZUj 1 = Z. (4.7) 

Since Z is a product of elements of {Uj^; v . . . , E^ 1 } and the sequence 
(E/fc+i, . . . , U r ) is AT-reduced, equality (4.7) can hold only when i = j and 
Ui and Z belong to the same cyclic subgroup of F n . Here, if Z contained 
Uj other than Uf 1 , we would have a contradiction with the fact that the 
sequence (E/fc+i, . . . , U r ) is iV-reduced. Hence, Z must be either trivial or 
a power of Ui. 

In a similar way, we can prove that only when i = j and Z is either 
trivial or a power of Ui can there be a cancellation in a subproduct of the 
form X^Y^^Z) Yj XJ Sji such that a part of X- i£ of length greater than 
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\Xi\ + \Xj\ cancels with a part of X- Sj£ of length greater than \Xi\ + \Xj\. 
Also, arguing similarly, we can show that such a cancellation cannot occur 
between X- i£ and X s - 3 £ or between X~ Sil and X- Sj£ . 

It then follows from (4.4) that after all cancellations in the product on the 
right-hand side of (4.5) are performed, some Xf , with £ f being sufficiently 
large (so £' > |PF|), must remain uncancelled. This contradiction to equality 
(4.5) completes the proof of Claim 1. ■ 

Since (p(W) G OrbAut F n {W), there is 7 G Aut F n such that (70 (p)(W) = 
W. Hence, upon replacing 7 o <p by <p, we may assume that 

(p(W) = W. (4.8) 

Under this assumption, we are going to prove: 

Claim 2. (p(F r ) = F r . 

Proof. Since (f\p r is a monomorphism by Claim 1, we must have, for such 
S G AutF r as in the proof of Claim 1, that the sequence ((</? o 5)(xi), 
(cp o S)(x 2 ), . . . , (<p o S)(x r )) is iV-reduced. As above, we write Ui for 
(ipo6)(xi), where i = 1 , 2 , ...,r. 

Clearly the set 5(X r ) = {£(# 1 ), S(x 2 ), . . . , <5(x r )} is also a basis of F r , 
which yields that W can be expressed in terms of 5(#i)’s. In this expression, 
say, Ws (i.e., Ws is a word in 5(X r ) ±:L ), every 5(xi), for i = 1 , 2 , . . . , r, must 
occur in Ws since W has rank r. Furthermore, since W has the minimum 
length over all words in OrbAut F n (W), the length of Ws with respect to 
the basis 5(X r ) must be at least \W\. 

Consider <p(Ws)- Obviously, <p(W$) = W by (4.8) and is a product of 
elements of {U^ 1 , . . . , E/^ 1 }, say, 

W = <p(Ws) = U%u£...U!z, ( 4 . 9 ) 

where qi G {1,2, .. . ,r}, Oi = ±1, and U^Uq^l ^ 1. Then m is equal to 
the length of Ws with respect to the basis 5(X r ) and hence m > \W\. Since 
the sequence (U\, U 2 , . . . , U r ) is A^-reduced, for every i some part of E7^, 
say, Vq.% must remain uncancelled in product (4.9) after all cancellations 
are made; that is, 

w = v q e ;v q ^--- v t- ( 4 - 10 ) 

Since m>\W\, each \V q . \ = 1 and hence 

X i = V Qi = Vqj = x qj if and ° nl y if Qi = 

where & = ±1. Replacing, if necessary, C /" 1 and —Qi by U Qi and 
respectively, we may assume 6' = 1 ; that is, 

X qi = V qi = Vq, = X qj if and OIll y Qi = Qy (4* 1 !) 

Here, if U qi = V qi for every i = 1 , 2 , ...,m, then the assertion follows 
immediately from (4.10) and (4.11). So suppose that there is some i such 
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that U q . V qi . Let j be the smallest index such that U q ' 3 ^ V q 3 . Then 
there is no cancellation in the product U q 3 Zl U q 3 , and no more than half of 

Uq 3 cancels in the product Uq- U q 3 +l because the sequence (t/i, • • • , U r ) 

is AT-reduced. Hence we must have 

u !i = 4 x Z ( 4 - 12 ) 

for some s £ {1, 2, , n}. 

By (4.9)-(4.12) and the fact that the sequence (C/i, C/ 2 ? • • • , U r ) is 
iV-reduced, there is a Whitehead automorphism x n +i) of F n + 1 

such that A C . . . jxf 1 }, x ^ e A, Xgf 1 £ A, and uji{W) = W. 

0 ■ 

This implies that the connected component C(x qj , W) of the Whitehead 
graph ${W) containing a vertex x q . does not contain x Qj 3 and that 
a Whitehead automorphism U 2 of F n + 1 given by W 2 = u) 2 (C(x qj , W), # n +i) 
fixes W. The fact U 2 (W) = W yields Xqf'iXq™ £ C(x 6 qj ,W). But then, 

O' O' 

a Whitehead automorphism us of F n defined by U 3 = us (C(x qj ,W),x qj ) 
sends W into ir x (W), where n x is an endomorphism of F n such that 
7 r Xq ( Xi ) = Xi if x qj ^ Xi G X and ir Xq , ( x Qj ) = 1. This contradiction to our 
assumption that W has the minimum length over all words in OrbAut F n {W) 
shows that there is no i such that U Qi ^ V qi , thus proving Claim 2. ■ 

It follows from Claims 1 and 2 that <p\ p r - F r F r is an automorphism of 

F r . Hence tp(x 1 ) is primitive in F r and therefore also in F n . This completes 
the proof of the theorem. ■ 
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Introduction to Part II 



This part is about affine algebraic geometry , a fascinating area of mathe- 
matics that studies polynomials and polynomial mappings. An interesting 
thing about this area is that most of the research here is focused on five 
or six outstanding problems. The statements of these problems are rather 
elementary and can be understood by an average high school student. How- 
ever, some methods that have been employed so far for attacking those 
problems are rather sophisticated, and, more importantly, they come from 
several different areas of mathematics, which stimulates additional interest. 

We start right off by giving the statements of five of these problems in the 
order of decreasing difficulty (in our opinion) and then give some very brief 
comments, leaving more details to the following chapters. We leave out one 
of the well-known problems, the Linearization conjecture, because we feel 
that we do not have sufficient experience with this problem ourselves. 

Throughout this part of the book, we denote by K[x i, . . . , x n \ the polyno- 
mial algebra in n variables over a field K of characteristic 0. More generally, 
we denote by AT[pi, . . . ,p n ] the subalgebra of K[x i, . . . , x n ] generated by 
polynomials pi , . . . ,p n . 

Problem 1 (the Jacobian conjecture [173]). If for n polynomials 
Pi, . . . ,p n £ K[x i, . . . , x n \ the corresponding Jacobian matrix is invertible, 
then K\pi, . . . ,p n ] = K[x u . . . 

This is the most famous of the five problems since it cuts across many 
different areas of mathematics, most notably algebra, algebraic geometry, 
complex analysis, and dynamical systems. There are several hundred pub- 
lished papers related to this problem, and probably a lot of unpublished 
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material, because some people may have felt that their research was not 
conclusive enough to be published. The problem is still open “even” for 
n — 2. It is a common belief that the conjecture is false for n > 3, but 
currently available computational resources are insufficient to find a coun- 
terexample. It seems plausible that a counterexample will be found within 
the next few years, when faster computers become available. 

On the other hand, the case n = 2 appears to be very tantalizing since 
there are so many partial results in the positive direction that one might 
get the feeling that a complete positive solution is very close. The other 
side of this coin is a growing belief that, since so many tools from so many 
different areas have been tried unsuccessfully, this might be an evidence of 
the conjecture being false for n = 2 as well. In the latter case, it is known 
that the degrees of polynomials in a counterexample should be over 100, 
which means it will be a while before we see such a counterexample. 

For a general survey and background on this problem, the reader is usu- 
ally referred to [39], which is, indeed, a good source for results up to 1981. 
However, in the last 20 years, a great number of new and interesting re- 
sults have been obtained, and we try to review at least some of them in 
the relevant chapter of this part of the book. We also refer the reader to 
a recently published monograph [122]. 

Problem 2 (the Cancellation conjecture of Zariski). Let R be a com- 
mutative algebra over a field of characteristic 0. Suppose R[x] is isomorphic 
to K[x i, . . . , z n +i]. Then R is isomorphic to K[x i, . . . , x n \ or, in geomet- 
ric language, let V C C n be a hypersurface. If V x C is isomorphic to 
a hyperplane in C n+1 , then V is isomorphic to a hyperplane in C n . 

Note that this problem, as well as many other problems in affine alge- 
braic geometry, admits both algebraic and geometric statements. We shall 
try to give both versions whenever possible, although we have to tell up 
front that our methods in this book are purely algebraic and, for the most 
part, elementary. When a problem is formulated in geometric language, the 
ground field is usually assumed to be the field of complex numbers. 

Unlike the previous problem, the Cancellation conjecture has been set- 
tled (affirmatively) in two nontrivial cases: n = 2 (Abhyankar, Eakin, and 
Heinzer [8]) and n = 3 (Miyanishi and Sugie [282] and Fujita [138]). It 
seems reasonable to expect a positive solution for larger values of n as well, 
but a proof most likely is not going to be easy. 

Problem 3 (Nagata’s problem). Is every automorphism of K[x i, . . .,x n \ 
tame? 

An automorphism of K[x i, . . . ,x n \ is called tame if it is a product of 
elementary and affine automorphisms. An automorphism of K[x i, . . . ,x n ] 
is called elementary if it fixes all variables but one. 

The answer to this problem is positive for n = 2; this is the classical 
Jung-van der Kulk theorem, which has been improved over the years. The 
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book [80] probably offers the most comprehensive exposition of this and 
related results. 

When this book was being prepared for publication, Shestakov and Umir- 
baev [342, 343] gave a (negative) solution of Nagata’s problem for n = 3 
by proving that a particular automorphism of K[x 1,^2, ^3] (incidentally, 
suggested by Nagata himself) is not tame. Furthermore, they gave an al- 
gorithm for deciding whether or not a given automorphism of K[x 1, £2, x z\ 
is tame. 

Their proof however cannot be carried on to higher dimensions, and 
there is still a possibility that the answer to Problem 3 might be positive 
for n > 4, although it seems to be a prevalent opinion that the answer 
should be negative for any n > 3. 

There are only a handful of papers on this very resistant problem, even 
though there is no lack of allegedly nontame automorphisms for any n > 3. 
The reason becomes clear after comparing this problem to the following 
Problem 4: it is extremely difficult to come up with invariants sensitive 
enough to distinguish between tame and stably tame automorphisms. 

Problem 4. Is every automorphism of K[x 1, . . . , x n \ stably tame? 

An automorphism a of K [aq, . . . , x n ) is called stably tame if the extension 
of a to some K[x i,...,Xjv], N > n, by a: Xi Xi, i > n, is a tame 
automorphism of K[x 1, ...,£#]• 

The answer to this problem is probably affirmative; at least, all known 
examples of allegedly nontame automorphisms turned out to be stably 
tame. However, the problem remains open for n > 3. 

To formulate the last of the five problems, we have to introduce some 
more notation. A polynomial p £ K[x 1, . . . , x n ] is called a coordinate poly- 
nomial , or simply a coordinate , if a(p) = x\ for some automorphism a 
of K[x 1, ...,x n \. Furthermore, for polynomials pi, . . . ,p m , by (pi, . . . ,p m ) 
we denote the ideal of K\x i,...,x n ] generated by pi,...,p m and by 
K[x 1 , . . . , x n ]/(pi, . . . ,p m ) the corresponding algebra of residue classes. 

Problem 5 (the Embedding conjecture of Abhyankar and Sath- 
aye). Let p e K[x i,...,x n ], n > 2. If the algebra K\x \, . . . , x n ]/{p) is 
isomorphic to K[xi , . . . , x n _i], then p is a coordinate, or, in geometric 
language, if a hypersurface V in C n is isomorphic to the hyperplane {x\ = 
0}, then it is equivalent to it. 

This problem was settled in the affirmative for n = 2 in [10] and [380], 
and it remains open for n > 3. 

In this part of the book, we address Problems 1 through 5 in Chapters 5 
through 8. 

In the last two chapters, 9 and 10, we treat coordinate polynomials and 
test polynomials, respectively. These topics were among the focal points of 
our own research in the last few years. Understanding (and algorithmically 
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recognizing) coordinate polynomials would be a key to some or all of the 
five problems above. 

Test polynomials, although not as immediately important as coordinate 
polynomials, appear to be a very attractive subject and have inspired a lot 
of research recently. They have been formally introduced in [124] as follows: 
a polynomial p G K[x \, . . . , x n ] is called a test polynomial if, whenever one 
has (p(p) = p for a mapping (p of K[x i, . . . ,x n ] into itself, it follows that 
ip is an automorphism. Polynomials (and sets of polynomials) with similar 
properties were earlier considered by Jelonek in [159]. In a more recent 
paper, Jelonek [163] showed that a generic (in a precise sense) polynomial 
of degree > n in C[xi, . . . , x n ] is a test polynomial for injective mappings. 
We discuss this and other results in Chapter 10. 

We conclude the Introduction by giving a statement of one of the most 
important results in affine algebraic geometry, the Abhyankar-Moh theo- 
rem [10, Main Theorem]. We have already mentioned one of the corollaries 
of this result in connection with Problem 5 above. Now we give the result 
itself: 

Theorem 6 ([10]). Let K be a field of characteristic 0, and let p = p(x), 
q = q(x) be two polynomials from K[x\ of degrees m > 1 and n > 1, 
respectively, such that K[x] = K\p,q\. Then either m divides n or n 
divides m. 

This “innocent-looking” statement turns out to be quite difficult to 
prove. Several different proofs were published over the years, but none 
of them were easy. This result now plays a key role in many arguments in 
affine algebraic geometry; it also plays an important role in this part of our 
book. 
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The Jacobian Conjecture 



In this chapter, we address the most famous of the problems about 
polynomial mappings: 

Problem 5.0.1 (the Jacobian conjecture). If for n polynomials 
Pi, . . . ,p n £ K[x i, . . . , a? n ], the corresponding Jacobian matrix is invertible, 
then K]p i, . . . ,p n ] = K[x i, . . . ,x n \. 

This is a very attractive problem because of the simplicity of its state- 
ment, which is accessible to a college student, and because there are so 
many ways to attack it. Commutative algebra, algebraic geometry, real 
and complex analysis, noncommutative algebra, algebraic topology, and 
differential algebra have all been used in papers related to the Jacobian 
conjecture. 

The Jacobian conjecture was first formulated by Keller in 1939 
(see [173]). It still remains an open problem aside from the trivial case 
n = 1. The case n = 2 is discussed in detail in the paper [5] by Abhyankar, 
where he proves the Galois case. He also shows that the conjecture is equiv- 
alent to the curves Pi = 0 having only one point at infinity in P 2 . He 
shows that they have at most two points at infinity, a result also proved by 
Makar-Limanov (unpublished). 

By using characteristic pairs and a computer search, Moh [283] proved 
the conjecture (n = 2) if deg(p^) < 100. Campbell [64] proved the Galois 
case for an arbitrary n > 2. 

Recently, Shpilrain and Yu [363] have proved that ip is an automor- 
phism if is invertible and for some automorphism i/j of K[x 1 ,^ 2 ] the 
composition fixes a nonconstant polynomial in K[x 1 ,^ 2 ]. 
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The case n — 2 is special also because only for n < 2 is the group of 
AT-algebra automorphisms of K[x i,...,a? n ] well-understood. For n = 2, 
a well-known theorem of Jung and van der Kulk says that the automor- 
phism group is a free product with amalgamation of two easily described 
subgroups. Many people think it is “almost certain” that the Jacobian 
conjecture is true for n = 2. For n > 2, the prevalent opinion is that the 
conjecture is false. 

Wang [397] proved a remarkable result: when deg(p^) < 2, the Jacobian 
conjecture is true. The fundamental paper of Bass, Connell, and Wright [39] 
reduces the Jacobian conjecture to the “cubic homogeneous case” . Namely, 
they proved the following result: 

Theorem 5.0.2. Suppose, for all n and all F E (K[x i, . . . , z n ]) n of the 
form F = X + N with N cubic homogeneous and J(N) nilpotent, that F 
is invertible. Then the Jacobian conjecture is true. 

Several alleged proofs of the Jacobian conjecture have been published, 
including [118], [330], [331], [332], and [334]. For a detailed history of the 
Jacobian conjecture up to 1981, see [39]. A survey of more recent results 
can be found in [122]. 

In the following sections, we give an exposition of our own contribution 
to this problem. 



5.1 Polynomial Retracts 

and the Jacobian Conjecture 

Let K[x,y\ be the polynomial algebra in two variables over a field K of 
characteristic 0. A subalgebra R of K[x,y\ is called a retract if it satisfies 
any of the following equivalent conditions: 

(Rl) There is an idempotent homomorphism (a retraction , or projection ) 
ip: K[x,y\ — > K[x,y\ such that <p(K[x,y]) = R. 

(R2) There is a homomorphism ip: K[x,y\ — >• R that fixes every element 
of R. 

(R3) K[x, y] = R 0 I for some ideal I of the algebra K[x, y]. 

(R4) K[x,y] is a projective extension of R in the category of AT-algebras. 
In other words, there is a split exact sequence 1 — > I -» K[x,y] —)• 
R-¥ 1, where I is the same ideal as in (R3) above. 

Examples: K\ K[x, y\, any subalgebra of the form K\p], where p E K[x, y] 
is a coordinate polynomial (i.e., K\p,q] = K[x,y] for some polynomial q E 
K[x, y]). There are other, less obvious examples of retracts: if p = x + x 2 y, 
then K\p] is a retract of K [x, y], but p is not coordinate since it has a fiber 
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{p = 0} that is reducible and therefore is not isomorphic to a line. Even 
less obvious examples are retracts generated by p = xy or p = x 2 — y 2 . 

The very presence of several equivalent definitions of retracts shows how 
natural these objects are. Later on, we shall also comment on a very natural 
geometric meaning of retracts. 

In [84], Costa has proved that every proper retract of K[x,y] (i.e., one 
different from K and K[x,y]) has the form K\p] for some polynomial 
p G K[x,y] (i.e., it is isomorphic to a polynomial AT-algebra in one vari- 
able). A natural problem now is to characterize somehow those polynomials 
p G K[x,y] that generate a retract of K[x,y]. Since the image of a retract 
under any automorphism of K[x,y] is again a retract, it would be reason- 
able to characterize retracts up to an automorphism of K[x,y] (i.e., up to 
a “change of coordinates”). We gave an answer to this problem by proving 
the following: 

Theorem 5.1.1 ([363]). Let K\p ] be a retract of K[x,y]. Then there is 
an automorphism of K[x,y] that takes the polynomial p to x + y • q for 
some polynomial q = q(x,y). A retraction for K[ip(p)\ is given then by 
x x + y - q; y ^ 0. 

Geometrically, our Theorem 5.1.1 says that (when K = C) every poly- 
nomial retraction of a plane is a “parallel” projection (sliding) on a fiber 
of a coordinate polynomial (which is isomorphic to a line) along the fibers 
of another polynomial (which generates a retract of K[x,y]). 

Our proof of this result is based on the well-known Abhyankar-Moh 
theorem (see Theorem 6). We note in passing that Theorem 5.1.1 also yields 
a characterization of retracts of a free associative algebra K(x,y) if one uses 
a natural lifting. In Section 5.2, we also make an observation on retracts of 
a polynomial algebra in arbitrarily many variables (Proposition 5.2.1). 

Theorem 5.1.1 yields another useful characterization of retracts of 
K[x,y]: 

Corollary 5.1.2. A polynomial p G K[x,y] generates a retract of K[x,y] 
if and only if there is a polynomial mapping of K[x,y] that takes p to x. 
The “if” part is actually valid for a polynomial algebra in arbitrarily many 
variables. 

Theorem 5.1.1 has several interesting applications, in particular to the 
Jacobian conjecture. We establish a link between retracts of K[x,y] and 
the Jacobian conjecture by means of the following: 

Conjecture 5.1.3 (Conjecture “R”). If for a pair of polynomials 
p, q G K[x, y] the corresponding Jacobian matrix is invertible, then K\p] is 
a retract of K [x, y\. 

This statement is formally much weaker than the Jacobian conjecture 
since, instead of asking for p to be a coordinate polynomial, we only ask for 
p to generate a retract, and this property is much less restrictive, as can be 
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seen from our Theorem 5.1.1. However, the point is that these conjectures 
are actually equivalent: 

Theorem 5.1.4. Conjecture “R” implies the Jacobian conjecture. 

There are several different ways of proving Theorem 5.1.4 using the char- 
acterization of retracts given in Theorem 5.1.1; we believe that the proof 
we give here (using Newton polygons) is particularly simple. 

As a corollary, we show that the Jacobian conjecture is equivalent to 
other (formally) weaker statements: 

Corollary 5.1.5. Either of the following claims is equivalent to the 
Jacobian conjecture. Suppose (p, q ) is a Jacobian pair. Then: 

(i) for some coordinate polynomial g , K[x,y] = K\p ] + (g), where (g) is 
the ideal of K[x,y] generated by the polynomial g; 

(ii) for some coordinate polynomial g, K[x,y] = K[g\ + (p). 

Corollary 5.3.1 in Section 5.3 also seems interesting since it improves 
several known partial results on two- variable Jacobian conjecture. 

Conjecture “R” has the following geometric interpretation. Suppose we 
have a polynomial p E C[x,y] that has a Jacobian mate. Then every fiber 
{p = c, c G C} is a nonsingular curve in the complex plane. Moreover, 
by a result of Kaliman [167], we can restrict our attention to the situation 
where every fiber of p is irreducible. 

If there is another polynomial fiber {h = Co} that is isomorphic to a line 
and intersects every fiber of p at exactly one point, then we can arrange 
a geometric projection of the plane onto the curve {h = Co} by sliding 
a point on a fiber {p = c} toward the intersection point of {p = c} with 
{i h = Co}. This geometric projection will also be algebraic (i.e., C[p] will be 
a retract of C[x,y] in this case). 

The only problem is to show that there is a fiber (isomorphic to a line) 
that intersects every fiber of p at exactly one point. In particular, we have: 

Corollary 5.1.6 (see [148]). Suppose (p is a polynomial mapping of 
C [x,y] with an invertible Jacobian matrix. If ip is injective on some line, 
then <p is an automorphism. 

We note here that the aforementioned result of Kaliman [167] calls for an 
example of a noncoordinate polynomial p £ C[x, y] that has a nonvanishing 
gradient and all of whose fibers are irreducible. A series of polynomials like 
that has been constructed in [18]. 

We also note that if we replaced the condition on a polynomial p(x, y) to 
have a Jacobian mate by a weaker condition to have a nonvanishing gradi- 
ent, then Conjecture “R” would not be true. The following counterexample 
has been communicated to us by A. van den Essen: p(x , y) = x+ (x+x 2 y) 2 . 

Another application of retracts to the Jacobian conjecture (although 
somewhat indirect) is based on the “<p°°-trick” familiar in combinatorial 
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group theory (see [383]). For a polynomial mapping p : K[x,y] K[x,y], 
denote by p°°(K [x, y]) = Hfeli <P fc (Jf[^, y\) the stable image of p. Then we 
have: 

Theorem 5.1.7. Let p be a polynomial mapping of K[x,y\. If the Ja- 
cobian matrix of p is invertible , then either p is an automorphism or 
'P°°(K[x,y]) = K. 

Our proof of Theorem 5.1.7 is based on recent results of Formanek [133] 
and Connell and Zweibel [83]. The result of Lemma 5.3.2, which is crucial 
for our proof of the theorem, seems to be of independent interest. 

Obviously, if p fixes a polynomial p G K[x,y], then p G p°°(K[x,y\). 
Therefore, we have: 

Corollary 5.1.8. Suppose p is a polynomial mapping of K[x, y] with an 
invertible Jacobian matrix. If p(p) = p for some nonconstant polynomial 
p G K[x,y], then p is an automorphism. 

This yields the following interesting reformulation of the Jacobian 
conjecture: if p is a polynomial mapping of K[x,y] with an invertible Ja- 
cobian matrix, then for some automorphism a, the mapping a • p fixes 
a nonconstant polynomial. 



5.2 Retracts of K[x, y] 

We start with the following proof: 

Proof of Theorem 5.1.1. Let K\p], p G K[x,y], be a retract of K[x,y] 
(note that, by a result of Costa [84], every retract of K[x, y) has this form). 

Let the corresponding retraction be given by p: x -» <71 (p); y — ► <72 (p) 
for some one- variable polynomials qi,q 2 . 

Since p is a retraction, polynomials <71 (p) , <72 (p) should generate K\p). 
Suppose qi and q 2 have degree n > 1 and m > 1, respectively. Then, 
by the Abhyankar-Moh theorem (Theorem 6), either n divides m or m 
divides n. Suppose deg(^i) = k • deg(^) for some integer k > 1. 

Then we make the following change of coordinates: x — > x = x — c* y k \ 
y y = p, where the coefficient c G K* is chosen so that in the polynomial 
Qi ~ c * Q 2 leading terms will cancel out. 

In these new coordinates, our retraction acts as follows: p:x->qi~c- 
Q 2 = 0i ; V -+ Q 2 = <72- Polynomials qi and q 2 are easily seen to be another 
generating set of K\p\, but the sum of degrees of qi and q 2 is less than that 
of qi and q 2 . 

Continuing this process, we shall eventually arrive at a pair of polyno- 
mials one of which is a constant c G K. Denote the other one by h; then 
we must have K[h\ = K\p] (i.e., h = c 1 • p + c 2 for some c\ G if*, c 2 e K). 
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Thus, we have shown that for some automorphism x/) £ Aut(K[x,y]) 
( “change of coordinates” ) the composition ipxjj takes x to c\ • p + c<i and 
y to c. It follows that c\ • xj){p{x,y)) + C 2 = x + (y — c) • q(x,y) for some 
polynomial q{x,y). Applying a linear transformation completes the proof. 

■ 

Proof of Corollary 5.1.2. 

(1) Suppose p £ K[x,y] generates a retract of K[x,y\. Then, by Theo- 
rem 5.1.1, for some automorphism xjj £ Aut (K[x,y]), the polynomial xp{p) 
has the form x + y • q(x , y). Let <p be a mapping of K[x, y] that takes x to x\ 
y to 0. Then <p(^(p)) = x. 

(2) We are going to prove the “if” part for a polynomial algebra 
K[x i, . . . , x n ] in arbitrarily many variables. 

Let ip(p) = X \ . Consider the following mapping of K[x i, . . . , x n \: xj ) : x\ — )> 
p; Xi -» 0, i = 1, . . . , n. Then 

V>(¥>(P)) = P ■ (5-1) 

Denote g = xjj( p . Then, by (5.1), g(p) = p, which means g fixes every 
element of K\p\. Also, it is clear that g(K[x i, . . . ,x n ]) = K\p\. Therefore, 
g is a retraction of K[x i, . . . , x n ] and K\p ] a retract. ■ 

We conclude this section with an observation on retracts of a polynomial 
algebra in arbitrarily many variables. 

Proposition 5.2.1. Let Rbe a proper retract ofK[x i, . . . , x n ] generated by 
polynomials pi, . . . ,p n , n > 2. Then pi, . . . ,p n are algebraically dependent. 

Proof. Let <p: K[x i,...,x n ] R be a retraction so that <p(R) = R. 
In particular, (p restricted to R is an automorphism of R. If polynomi- 
als pi, . . . ,p n were algebraically independent, then a result of [83] would 
imply that ip is an automorphism of K\x i, ...,x n ] as well. In that case, we 
would have R = K[x i, . . . , x n ], which means R is not a proper retract and 
hence a contradiction. ■ 

We note that it is an open problem whether or not any retract of 
K[x i,...,x n ], n > 3, can be generated by algebraically independent 
polynomials. This problem is related to (some forms of) the cancellation 
problem — see [84] for discussion. 



5.3 Applications to the Jacobian Conjecture 

First we prove that our conjecture “R” implies the Jacobian conjecture. 

Proof of Theorem 5.1.4. Let <p: x p(x,p); y -* q(x,y) be a 
polynomial mapping of K[x, y] with an invertible Jacobian matrix. 

If we assume that our conjecture “R” is true, then p(x,p) generates 
a retract of K[x,y]\ hence, by Theorem 5.1.1, for some automorphism xjj £ 
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Aut (K[x, p]), polynomial ^(p) has the form x + y • h(x , y). Therefore, upon 
combining ip with if; if necessary, we may assume that p(x , y) itself has this 
form. 

Now we appeal to a result of [17] (see also [201]) concerning Newton 
polygons of a Jacobian pair (p(x, p), q(x , y)). The Newton polygon of a poly- 
nomial / = f(x,y) = Y^ a ijX l y j is the convex hull of {{i,j) \ aij ^ 

0}u{(0,0)}. 

The result of [17] we need is that if (p(x,y),q(x,y)) is a Jacobian pair, 
but p: x -» p(x, p); y — > q(x, y) is not an automorphism, then Newton 
polygons of p(x, y) and q(x , y) are radially similar. 

Now, by way of contradiction, suppose ip is not an automorphism. Look 
at the Newton polygon of p(x , y). We see that it has an edge of length 1 — 
namely, the one between the vertices (0,0) and (1,0). It follows that the 
similarity ratio for Newton polygons of p(x , y) and q(x , y) is an integer; in 
particular, q(x , y) has the form J2i=i c i x% + V * f( x > V) f° r some c* £ AT, 
c k ± 0. 

Then replace the pair (p, q) with (p, q — c k p k ). This new pair clearly has 
the same properties as (p, q) does: it is a Jacobian pair, but the corre- 
sponding mapping is not an automorphism. However, the highest degree of 
monomials of the form x m in the second polynomial has been decreased. 

Therefore, we can repeatedly apply our argument (note that p(x, y) does 
not change) until we get a pair (p, p), where g has no monomials of the 
form x m . But, in that case, the Newton polygon of g has no edges along 
the x-axis; hence Newton polygons of p and g cannot be radially similar. 
This contradiction completes the proof of Theorem 5.1.4. ■ 

Proof of Corollary 5.1.5. 

(i) Without loss of generality, we may assume that both p and g have 
zero constant term. Moreover, upon applying an automorphism to all 
polynomials under consideration if necessary, we may assume g = x. 

First we show that if K[x,y] = K\p\ + (x), then the sum is actually 
direct. 

By way of contradiction, suppose we have 

m 

x-u = Y,Ci-p' (5.2) 

i= 1 

for some nonzero polynomial u = u(x,y) and constants c* £ K. Since 
the left-hand side of (5.2) is divisible by x , the right-hand side should be 
divisible by x, too. This is only possible if p itself is divisible by a;, but, 
in that case, the Newton polygon of p would not have an edge along the 
p-axis, which contradicts p having a Jacobian mate (see, e.g., [201]). 

Thus, we have shown that K[x,y\ = K\p] 0 (x). By the definition (R3) 
of a retract, this implies K\p] is a retract of K[x, y]. Therefore, by our 
Theorem 5.1.4, K\p,q\ = K[x,y]. 
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(ii) Arguing as in (i), we get p - u = YT=i c i ' x% - I n this case, p cannot 
depend on p, so p = p(x); but then the equality K[x, y] = K[x] + (p) is not 
possible. 

Thus, K[x, y] = K[x] 0 (p). This clearly implies p = c-y for some c £ K* . 
The result follows. ■ 

Corollary 5.3.1. Let K be an algebraically closed field, and let ( p,q ), 
p,q £ K[x,y], be a Jacobian pair. Suppose p has the form p = x + g 
for some g £ K[x,y] that is divisible by a homogeneous polynomial. Then 
K\p, q] = K[x,y], 

Proof. If p is linear, then we are done. Suppose p is nonlinear, and suppose 
g is divisible by a nonconstant homogeneous polynomial h. 

If h is divisible by x, then p itself is divisible by x. In that case, the New- 
ton polygon of p does not have an edge along the p-axis, which contradicts 
p having a Jacobian mate (see [201]). 

If h is divisible by y , then K\p] is a retract of K[x,y\. If not, then there 
is c £ K* such that the homomorphism x — )> p; y cp takes h to 0. This 
is obviously a retraction, so again K\p\ is a retract of K[x,y]. 

Applying Theorem 5.1.4 yields the result. ■ 

Van den Essen and Tutaj [125] have shown that if a Jacobian pair (p, q) is 
of the form + P+/ 12 ), where both hi and h<i are homogeneous polyno- 

mials, then K\p, q] = K[x, y\. It is notable that our Corollary 5.3.1 not only 
relaxes the condition on the form of polynomials but, most importantly, our 
condition is imposed on one polynomial only. 

To prove Theorem 5.1.7, we need the following result of independent 
interest: 

Lemma 5.3.2. Let <p: x — >• p(x,y); y — > q(x,y) be a polynomial mapping 
of K[x,y] with an invertible Jacobian matrix. Suppose (p(K[x,y]) contains 
a coordinate polynomial. Then ip is an automorphism. 

Proof. Upon composing p with an automorphism if necessary, we may 
assume that x £ (p(K[x, y]). This implies K [p, q , x] = K\p , q] and, therefore, 
K(p, q , x) = K(p, q), where K(p , q) is the quotient field of K\p, q\. 

On the other hand, by a result of [133], (p, q) being a Jacobian pair 
implies K(p , q , x) — K(x, y). 

Therefore, we have K(p,q) = K(x,y), which by Keller’s theorem [173] 
implies K\p, q] = K[x, y\. ■ 

Proof of Theorem 5.1.7. Suppose p is not an automorphism. Then, by 
Lemma 5.3.2, p(K[x,y]) contains no coordinate polynomials. In partic- 
ular, the degree of any polynomial in p(K[x,y]) is at least 2. By using 
inductive argument, we are going to show now that the degree of any 
nonconstant polynomial in p k (K[x,y]) is at least ( k + 1); this will imply 
<p°°(K[x,y]) = K. 
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Consider an algebra (p k (K[x , y]) for some k > 1. Since ip is injective (this 
is ensured by the Jacobian condition), polynomials (p k (x ) and <p k (y) are 
algebraically independent. 

If p restricted to ip k {K[x 1 y)) were an automorphism of p k (K[x , y ]), then, 
by a result of [83], ip would be an automorphism of K[x,y\, contrary to our 
assumption. (Note that ip k {K{x , y]) is invariant under <p since, by induction, 
cp k (K[x, y]) C ^ fc_1 (K[a;,y]) implies ip k+1 {K[x,y\) C cp k (K[x,y]).) Hence, 
p | yk(K[x, y }) i s no ^ an automorphism of (p k (K[x, ?/]), and our previous ar- 
gument yields that ip k + l (K[x, y]) contains no coordinate polynomials of 
ip k (K[x, y]). In particular, the degree of any polynomial in ip k + l (K[x , y]) is 
greater than that of any coordinate polynomial in cp k (K[x, y]). This com- 
pletes the induction. ■ 

In the case K = C, a more complicated (geometric) proof of the result 
of Theorem 5.1.7 was given by Kraft [184], who also proved that if a poly- 
nomial mapping <p of C[x, y\ is not birational (i.e., ip does not induce an 
automorphism of the quotient field C(x,y)), then the stable image of <p is 
a retract of C[^, y]. This yields a natural question — is the same true for 
any polynomial mapping? (See [383].) 




6 

The Cancellation Conjecture 



In this chapter, we address another famous problem: 

Conjecture 6.0.1 (the Cancellation conjecture). Let R be a commu- 
tative algebra over a field of characteristic 0. Suppose R[x] is isomorphic 
to K[x i, . . . , x n +i]. Then R is isomorphic to K[x i, ...,x n \, or, in geomet- 
ric language: let V C C n be a hypersurface. If V x C is isomorphic to 
a hyperplane in C n+1 , then V is isomorphic to a hyperplane in C n . 

Here we are going to establish a connection between this problem and 
another well-known problem, the Embedding conjecture of Abhyankar and 
Sathaye (see Chapter 8), by introducing an “intermediate” conjecture 
(a precise meaning of this will be clear below). For future reference, we 
shall call it “the Stable coordinate conjecture” . We have to introduce some 
more notation first. 

We focus here on the case K = C. Any collection of polynomials 
pi, • • • ,Pm from C[xi, . . . , x n ] determines an algebraic variety {pi = 0, i = 
1, . . . , m} in the affine space C n . We shall denote this algebraic variety by 
V(pu...,p m ). 

We say that two algebraic varieties V(pi, . . . ,p m ) and V(qi, . . . , qk) are 
isomorphic if the algebras of residue classes C[#i, . . . , x n \/ (pi, . . . ,p m ) and 
C[#i, . . . , x n \/ (qi, ...,qk) are isomorphic. Here (pi, . . . ,p m ) denotes the 
ideal of C[a?i, . . . , x n \ generated by pi, . . . ,p m . Thus, the isomorphism that 
we consider here is algebraic, not geometric (i.e., we actually consider 
isomorphism of what is called affine schemes). 

On the other hand, we say that two algebraic varieties V^Pi, • • • ,p m ) 
and VX<Zi,. . . ,<Zfc) are equivalent if there is an automorphism of C n that 




6. The Cancellation Conjecture 81 



takes one of them onto the other. Algebraically, this means there is an 
automorphism of C[#i, . . . , x n \ that takes the ideal (pi, . . . ,p m ) to the ideal 
(■ qi,---,Qk >• 

Furthermore, a variety equivalent to V x C is called a cylinder , a variety 
of the form {p = 0} is called a hypersurface, and a hypersurface equivalent 
to {xi = 0} is called a hyperplane. In particular, a cylindrical hypersurface 
in C n is a variety of the form {p(x i, . . . , x m ) = 0}, where m < n. Then we 
have: 

Conjecture 6.0.2 (the Embedding conjecture of Abhyankar and 
Sathaye). If a hypersurface V(p) in C n is isomorphic to the hyperplane 
V(xi), then it is equivalent to it. 

Conjecture 6.0.3 (the Stable coordinate conjecture). Let V C C n 

be a hypersurface. If V x C is equivalent to a hyperplane in C n+1 , then V 
is equivalent to a hyperplane in C n , or, in purely algebraic language: if p = 
p(x i, . . . , x n ) and (p(p ) = x\ for some automorphism ip of C[x \, . . . , # n +i], 
then also a(p) = x\ for some automorphism a of C[xi, . . . , x n \. 

The focus of this chapter is actually on Conjecture 6.0.3. It looks similar 
to the Cancellation conjecture 6.0.1; the difference is that isomorphism here 
is replaced by equivalence. 

All the three conjectures seem to be related; however, it is not so easy 
to pinpoint precise relations among them. Here we start with the following 
simple observation. 

Proposition 6.0.4. 

(a) Conjectures 6.0.1 and 6.0.2 together imply Conjecture 6.0.3. 

(b) Conjectures 6.0.3 and 6.0.2 together imply Conjecture 6.0.1. 

(c) Conjecture 6.0.3 is true for n = 2 and 3. 

Our proof of part (c) in the case n = 3 is based on deep results cited 
above, whereas the case n = 2 can also be handled by somewhat more 
elementary methods based on a result of [65]. 

A much less trivial result is the following straightforward implication: 

Theorem 6.0.5 ([236]). For each particular n, the Stable coordinate 
conjecture implies the Cancellation conjecture. 

Conjecture 6.0.3 also motivates the following generalization: 

Problem 6.0.6 (Stable equivalence problem). Are any two stably 
equivalent polynomials equivalent? Or, in geometric language: is it true 
that equivalence of two cylindrical hypersurfaces V(p) x C and V (q) x C 
in C n+1 implies equivalence of their bases V(p) and V(q) in C n ? 

A weaker version of this problem is still quite hard: 
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Problem 6.0.7. Is it true that equivalence of two cylindrical hypersurfaces 
V(p) x C and V(q) x C in C n+1 implies isomorphism of their bases V(p ) 
and V(q) in C n ? 

A generalization of Conjecture 6.0.3 in another direction is: 

Problem 6.0.8. Can a cylinder in C n be equivalent to a noncylinder in 
C n under an automorphism of C m , m> n? 

Problem 6.0.6 is, in our opinion, the most interesting one, and it probably 
holds the key to many mysteries of affine algebraic geometry. 

We show here that the answer to Problem 6.0.6 is affirmative in the two- 
variable case (Theorem 6.0.11). We also show that for n > 3, the answer 
is affirmative for two rather large classes of polynomials (Proposition 6.0.9 
and Theorem 6.0.10). The first result is just a simple observation: 

Proposition 6.0.9. Let p = p(x i, . . . , x n ) be of the form x f* 1 x^f n + 

c(j) * x l f^ . . .x l n^\ where c(j ) are some coefficients , all Mi > 0, all 

monomials under the sum are different from x^ 1 x^ n , and, for every 

k, j, one has ik{j) < M&. Then any polynomial q = q(x i, . . . ,x n ) stably 
equivalent to p is equivalent to it, or, in geometric language: for any hy- 
persurface V(q) in C n , equivalence ofV(p) x C and V(q) x C k in C n+k 
implies equivalence of V (p) and V (q) in C n . 

We note that if p is a coordinate in C[aq, . . . , x n \, then p cannot possibly 
satisfy the condition of Proposition 6.0.9 since, by a result of Hadas [149], 
all vertices of the Newton poly tope of a coordinate polynomial must be 
on coordinate hyperplanes. Therefore, Proposition 6.0.9 does not settle 
Conjecture 6.0.3. 

On the other hand, it should be pointed out that, even if a given polyno- 
mial p does not satisfy the condition of Proposition 6.0.9, there might be 
an automorphic image of p that does, and then the result will hold for p 
as well. For example, p(x , y) = (x + y) 2 y 2 does not have the form required 
in Proposition 6.0.9, but it has an automorphic image q(x,y) = x 2 y 2 that 
does. We discuss the two- variable case in more detail after the proof of 
Proposition 6.0.9. 

Another, more interesting class of polynomials for which Problem 6.0.6 
has the affirmative answer is given by the following: 

Theorem 6.0.10 ([365]). Suppose a polynomial p = p(x i, . . . , x n ) has the 
following property: if ^(p) = p for some injective polynomial mapping xJj of 
C[a?i, . . . , x n \, then 'ip must be an automorphism of C[xi, . . . , x n ]. Then any 
polynomial q = q(x \, . . . , x n ) stably equivalent to p is equivalent to it. 

Polynomials that satisfy the condition of Theorem 6.0.10 are called test 
polynomials for injective polynomial mappings — see [124]. Jelonek [163] 
showed that a generic polynomial of degree > n in C[#i, . . . , x n ] is a test 
polynomial for injective mappings. The statement “a generic polynomial of 
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degree d has a property W” means that there exists a Zariski open dense 
subset U C Hd of the set 7 id of all polynomials of degree d such that every 
element of U has the property W. Thus, the class of polynomials covered 
by Theorem 6.0.10 is really large. Still, it does not include any coordinate 
polynomials, as was observed in [124]. 

Then, we have a complete solution of Problem 6.0.6 in the two-variable 
case: 

Theorem 6.0.11 ([236]). If two polynomials p, q E K[x,y] are stably 
equivalent, then they are equivalent . 

We can also answer Problem 6.0.8 in the affirmative: 

Theorem 6.0.12. For any k > 1 and n > k + 2, there is a variety V x C k 
in C n and a noncylinder U in C n such that U is equivalent to V x C k in 
C 2n . 

The simplest example (that we know) illustrating Theorem 6.0.12 would 
be U = {x(l -f xy + z 2 ) = 1} in C 3 and V = {xy = 1} in C 2 . The varieties 
U and FxC are inequivalent in C 3 but are equivalent in C 6 (in fact, they 
are equivalent even in C 4 ). 



6.1 Equivalence and Stable Equivalence 

Here we give proofs of the statements from the previous section. 

Proof of Proposition 6.0.4. We start with part (a). We are going 
to prove this statement in the following form: if both Conjectures 
6.0.1 and 6.0.2 are true for any n, then for any k> 1, whenever V x C k is 
equivalent to a hyperplane in C n+fc , one has V equivalent to a hyperplane 
in C n . 

Let V = V(p), p = p(x i, . . . , x n ), and suppose that p is a coordinate in 
C[^i, . . . ,x;v], N = n + k. Then, in particular, the hypersurface {p = 0} 
in is isomorphic to C^ -1 . If Conjecture 6.0.1 is true, this implies that 
{p = 0} in C n is isomorphic to C n_1 . Now if Conjecture 6.0.2 is true, this 
implies that p is a coordinate in C[#i, . . . , x n ], and hence V is equivalent 
to a hyperplane in C n . 

For part (b), let V = V(p), p = p(x i,...,x n ), and suppose that the 
hypersurface {p = 0} in C N , N > n, is isomorphic to C^ -1 . If Conjec- 
ture 6.0.2 is true, this implies that p is a coordinate in C[#i, ...,£#]• Then 
Conjecture 6.0.3 implies that p is a coordinate in C[#i, . . . , x n \ as well. In 
particular, the hypersurface {p = 0} in C n is isomorphic to C n_1 . 

For part (c), we start with n — 2. Let V = V(p), p = p(x i,x 2 ), and 
suppose that for some N > 3, p is a coordinate in C[xi , . . . , xn]. Then, in 
particular, the hypersurface {p = 0} in C N is isomorphic to C^ -1 . Then, by 
the result of Abhyankar, Eakin, and Heinzer [8] and Miyanishi [281] cited 
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in the Introduction, the hypersurface {p = 0} in C 2 is isomorphic to C. 
This implies, by the Abhyankar-Moh theorem [10], that p is a coordinate 
in C[xi,x 2 ]. 

Now we get to n = 3. The proof here is similar. Let V = V(p), p = 
p(x 1,^2, ^3)- If p is a coordinate in C[#i, . . . ,xn] for some N > 3, then 
also for any c £ C, p — c is a coordinate in C[#i, . . . ,##]. Therefore, for 
any c G C, the hypersurface {p = c} in is isomorphic to C^ -1 . Then, 
again for any c G C, the hypersurface {p = c} in C 3 is isomorphic to C 2 
by the result of Fujita [138] cited in the Introduction. Now this implies, by 
a recent result of Kaliman [169] (see also [171]), that p is a coordinate in 
C[x\, #2, £3], and hence V is equivalent to a hyperplane in C 3 . ■ 

Proof of Theorem 6.0.5 One of the equivalent formulations of the Can- 
cellation conjecture is (see [122, p. 54]): for every locally nilpotent 
derivation V of the algebra K[x 1 , . . . , x n \ with a slice s, the kernel Ker V 
is isomorphic to K[x 1 , . . . ,x n _i]. By Proposition 2.1 of Wright [405], the 
latter property is equivalent to s being a coordinate, i.e., an automorphic 
image of x\. 

Thus, we start with an arbitrary locally nilpotent derivation V of the 
algebra K[x x,...,x n ] with a slice s, and we want to prove that s is 
a coordinate in K[x 1 , . . . , x n ]. 

Extend V to K[x ±, . . . ,x n +i] by D(x n +i) = 0. Then KevV in 
K[x 1 , . . . , x n +i] is K[x 1 , . . . , £nP[#n+ 1], where K[x 1 , . . . , x n ] v denotes 
KeiV in K[x i,...,x n ]. Since s is transcendental over K[x i,...,^] 11 , 
we have K[x \, . . . ,£ n p[a; n +i] isomorphic to K[x \, . . . ,x n ] v [s\. The lat- 
ter algebra is equal to K[xi , . . . , x n \ by the result of Wright [405, 
Proposition 2.1]. 

Thus we get K[xi , . . . , £ n +iP isomorphic to K[x 1 , . . . , x n ], which implies 
that s is a coordinate in K\x 1 , . . . , # n +i]- Since we are under the assump- 
tion that the Stable coordinate conjecture holds for this particular n, we 
conclude that s is a coordinate in K[x \, . . . , x n ], and therefore the Cancel- 
lation conjecture holds for the same n. ■ 

Before we get to the proof of Theorem 6.0.11, we have to do some 
preliminary work. 

Let p, q G K[x , y]; <p(p) = q for some automorphism (p of K[x , y, 2 , . . . ]. 

Let <p(x) = u = u(x , y, 2 , . . . ), <p(y) = v = v(x, y, z, . . . ). We are going 
to prove a stronger statement (Proposition 6.1.1 below) that will imply 
Theorem 6.0.11. 

We call a pair (u, v ) of polynomials z-reduced if the sum of z-degrees of 
the two polynomials cannot be reduced by either a (nondegenerate) linear 
transformation or a transformation of one of the following two types: 

(i) (u, v) — > (u + p • v k , v) for some p G K* \ fc > 2; 

(ii) (u, v) — > (u,v + p - u k ). 

When proving Theorem 6.0.11, we can assume, without loss of generality, 
that the pair ( u(x , y, z, . . . ), v(x, y, z, . . . )) is z-reduced. 
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If ( u , v) is a pair of two-variable polynomials such that the sum of their 
degrees cannot be reduced by a transformation of one the above types, then 
we call this pair elementary reduced. 

Proposition 6.1.1. Let p G K[x,y] be a two-variable polynomial. 
Let (u(x, y, z, . . . ), v(x, y, z, . . . )) be a z-reduced pair of algebraically in- 
dependent polynomials such that both of them actually depend on z. 
Then , for any N G Z+, there is a polynomial w = w(x, y) such that 
d eg(p(u(x, y, w, 0, . . . , 0), v(x, y,w, 0, . . . , 0))) > N. 

In the proof of Proposition 6.1.1, we shall write just u(x,y,z ) and 
v(x, y, z) instead of u(x , y, z, 0, . . . , 0) and v(x, y,z, 0, . . . , 0) to simplify the 
notation. First we prove 

Lemma 6.1.2. Let u(x,y,z) and v(x,y, z) be algebraically independent. 
For any Mg2 + and m,n> M, there is c G K such that u(x , y, x m y n + c) 
and v(x, y, x m y n + c) are algebraically independent. 

Proof. Recall that polynomials /i, . . . , / m G K[x i, . . . , x n \ are alge- 
braically dependent over K if and only if the Jacobian matrix D(/i, . . . , f m ) 
has rank smaller than m. 

Assume, by way of contradiction, that for all c G K the polynomials 
u(x,y,x m y n + c),u(x,y,x m y n + c) G K[x,y ] are algebraically dependent. 
This means that for all c G AT, the matrix 

D{u{x,y,x m y n + c), u(x, y, x m y n + c)) , i 

D(u, v) | (x,y,x rn y n +c) * I ^ 

\mx m ~ 1 y n 

has rank at most 1. Then for all c G K the 2x3 matrix D(u , v)\( x ,y,x m y n +c) 
has rank at most one, which means that all its 2 x 2 minors are 0. Using the 
fact that for all a, b G K the map from K to K definied by c a m b n + c is 
surjective, this implies that for all a, 6, c G K all 2x2 minors of Z?(u, v)\( a ^, c ) 
are 0. Since K is an infinite field, this in turn implies that all 2 x 2 minors 
of D(u,v) are 0. Thus, the rank of D(u,v) is at most one and therefore 
u and v are algebraically dependent over AT, a contradiction. ■ 

Proof of Proposition 6.1.1 Since both u(x,y,z) and v(x,y,z) actually 
depend on z and the pair is z-reduced, we can find m, n > 2N and 
c G K such that u(x, y, x m y n + c) and v(x, y, x m y n + c) are algebraically 
independent (by Lemma 6.1.2) and elementary reduced. 

Now we use a result of Shestakov and Umirbaev from [342] which implies, 
in particular, that, if two polynomials r(a;, y) and s(x, y) of degree > 2N are 
algebraically independent and elementary reduced, then every nonconstant 
polynomial in the algebra K[r, 5 ] has degree at least N + 2. This completes 
the proof of Proposition 6.1.1. ■ 

Now we can get to the 
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Proof of Theorem 6.0.11 Recall that <p(x) = u = u(x , y, z, . . . ), ip(y) = 
v = v(x,y, z, . . . ). Upon applying an automorphism of K[x,y\ to both 
p(x, y) and q(x , y) if necessary, we may assume that u(x, y, z, . . . ) and 
v(x, y, z, . . . ) are z-reduced. Now we have several cases. 

Case 1. Both u(x, y, z, . . . ) and u(x, y, z, . . . ) actually depend on z. Then 
we can apply Proposition 6.1.1 to get a contradiction in this case. 

Case 2. Say, v(x, y, z, . . . ) actually depends on z, whereas u(x, y, z, . . . ) 
does not. Let x m y n be the highest monomial in p(x,y) with respect to 
“lexdeg” ordering with y > x. This monomial will contain the highest 
power of z after we plug in u for x and v for y. This highest power of z then 
cannot cancel out in p(u, u). Therefore, p(u,v) will depend on z, contrary 
to the assumption p(u, v) = q(x , y). 

Case 3. Neither u(x 1 y, z, . . . ) nor v(x, y, z, . . . ) depend on z. If there are 
other variables that either u(x, y, z, . . . ) or v(x, y, z, . . . ) depend on, then 
we find ourselves in Case 1 or 2 above. If not, then there is nothing to 
prove because the restriction of if to K[x, y] must be an automorphism of 
K[x,y\. 

This completes the proof of Theorem 6.0.11. ■ 

Remark 6.1.3. The crucial technical tool in our proof of Theorem 6.0.11 
was Shestakov-Umirbaev’s result from [342] that bounds (from below) the 
degree of polynomials in the subalgebra of K[x, y] generated by two given 
polynomials. This is (philosophically) similar to “small cancellation” ideas 
in combinatorial group theory (see e.g. [227]). We note however that in 
commutative algebra, these ideas cannot be simply carried on to higher 
dimensions as the following example shows. 

Let ip: x -* u = x — yt 2 z 2 , y — » v = 1 + tz 2 , z — >> r = z 2 , t s = 
—xt + yt 2 + yt 3 z 2 . Let p = p(x,y, z,t) = xy + zt. Then ip(p) = x, i.e., 
x G K[u, u, r, s], even though the degrees of u, v, r, s are at least 2. Similar 
examples can be constructed with arbitrarily high degrees of u, v, r, s. 

This example therefore makes it appear likely that our proof of The- 
orem 6.0.11 might be difficult to carry on to higher dimensions, but, of 
course, this does not mean that the result itself does not hold. 

We now get to our partial results on Problem 6.0.6 for arbitrary n > 2. 

Proof of Proposition 6.0.9. Let <p(p) — q for some automorphism (p of 
C[xi, . . . ,£jv], N > n. Then all Xi with i > n have to cancel out in (p{p). 

However, because of the presence of the “dominating” monomial x^ 1 

in p, if there is some Xi with i > n in some of the <p(xj), l < j < n, then 
this Xi will not cancel out in <p(p). Therefore, none of the (p(xj), 1 < j < n, 
depend on any x^ i > n, in which case the restriction of <p to C[^i, . . . , x n \ 
must be an automorphism of C[x \, . . . , x n \. ■ 

Remark 6.1.4. As we have mentioned before, even if a given polynomial 
p does not satisfy the condition of Proposition 6.0.9, there might be an 
automorphic image of p that does, and then the result will hold for p as well. 
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Here we note that, in the case where p is a two- variable polynomial over C, it 
is algorithmically possible to find out whether some automorphic image of p 
has the form required by Proposition 6.0.9. Indeed, Wightwick [401] showed 
that, if some automorphic image of p has lower degree than p does, then 
there is a single elementary automorphism that reduces the degree of p. At 
the same time, it is easy to see that no elementary automorphism can reduce 
the degree of a polynomial that satisfies the condition of Proposition 6.0.9. 
Thus, given a polynomial p, we keep applying elementary automorphisms 
that reduce the degree until the degree becomes irreducible. Then we check 
if there is a linear or elementary automorphism preserving the degree of 
the obtained polynomial that produces a polynomial in the form required 
by Proposition 6.0.9. For more details, we refer to [401]. 

We conclude this section with 

Proof of Theorem 6.0.12. First we give a proof for k = 1, n = 3 to 
simplify the notation; then we explain how this proof generalizes easily to 
arbitrary k > 1 and n > k + 2. 

We are going to show that U = {x(l 4- xy + z 2 ) = 1} is isomorphic to 
W = {xy = 1} in C 3 . The latter variety obviously is of the form V x C, 
whereas the former is not. 

It will be technically more convenient to write algebras of residue 
classes as “algebras with relations”; i.e., for example, instead of 
C[xi,...,x n ]/(p(a;i,...,x n )) we shall write (xi,...,x n | p{x ll . . . , x n ) = 
0 ). 

Now we get the following chain of “elementary” isomorphisms: 

(x,y,z | x{l + xy + z 2 ) = 1} 

= (x, y, z | x(l + xy + z 2 ) = 1, xy{ 1 + xy + z 2 ) = y) 

= (x,y,z \ x + x 2 y + xz 2 = 1, xy 4- x 2 y 2 + xyz 2 = y) 

= (x, y, 2 , u | x + x 2 y + xz 2 = 1, xy + x 2 y 2 + xyz 2 = y, u = xy) 

= (x, p, z, u | x + xu + xz 2 = 1, y = u -f u 2 + uz 2 , u = xy) 

= (x, z, u | x + xu + xz 2 = 1, u = x(u + u 2 + uz 2 )) 

= (x, y, z | x + xy + xz 2 = 1, y = x(y + y 2 + yz 2 )) 

= (x, y, z | x + xy + xz 2 = 1) 

= (x,y,z | x(l + y + z 2 ) = 1) 

- (x,y,z | xy = 1). 

Thus, U = {x(l + xy + z 2 ) = 1} is isomorphic to W = {xy = 1} 
in C 3 . Upon replacing U with U m = {x(l + xy + z\ + • • • + z^) = 1}, 
where zi, . . . , z m are variables, and using the same chain of “elementary” 
isomorphisms, we get examples, for any m > 1, of noncylinders U m iso- 
morphic to W = {xy = 1} in C m+2 . Then, upon replacing U m with 
Um,r = {x • ti t r • (1 + xy 4- z 2 -}- • • • + z^) = 1} and W with 
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W r = {xy-ti t r = 1}, where ti, . . . , t r are variables, and using the same 

chain of isomorphisms, we get the isomorphism of C/ m?r to W r in C m+2+r 
for any m > 1, r > 0. 

To show that, for instance, the polynomial x(l + xy + z 2 -\ b z^) is 

not equivalent to any polynomial in less than (m + 2) variables, it is clearly 

sufficient to show the same for the polynomial q = 1 + xy + z\ H b z^- 

The gradient of the latter polynomial is (y, x, 2zi , . . . , 2 z m ). Now the “chain 
rule” for partial derivatives shows that, if we apply an automorphism p to 
q , then the gradient of p(q) must contain all the variables x,y,zi,...,z m . 
Then the same is true for the polynomial p(q) itself. 

Finally, to get the equivalence claimed in the statement of Theorem 6.0.12 
out of the isomorphism above, we just recall (see, e.g., [364, Corol- 
lary 1.2]) that if two algebraic varieties V(pi , . . . ,p m ) and F(<zi, . . . ,<&) 
in C n are isomorphic, then the varieties F(pi, . . . ,p m , £ n +i, . . . , X 2 n ) and 
qk, ^n+i) • • • , X 2 n) in C 2n are equivalent under a (tame) automor- 
phism of C 2n . ■ 



6.2 Around Danielewski’s Example 

In Danielewski’s example [88] of two isomorphic cylindrical hypersurfaces 
V(p) x C and V(q) x C with nonisomorphic bases V(p) and V(q) in C 3 , we 
have p = p(x, y^z) — xy — z 2 q — q(x , y, z) = x 2 y — z 2 + 1. We start 
this section by giving an explicit algebraic isomorphism (due to P. Russell) 
for cylinders in Danielewski’s example. This argument also establishes the 
fact that Danielewski’s cylinders are isomorphic over any ground field of 
characteristic 0, not just over C. We are grateful to P. Russell for kindly 
permitting us to use his observation. 

Remark 6.2.1 (Russell, unpublished). Let p = p(xi,yi,zi) = x\y\ - 
z\ + 1; q = q(x 2 ,y 2 ,Z 2 ) = x\y 2 — z\ + 1, and let K be an arbitrary 
field of characteristic 0. The following mapping p from K[xi,yi,zi,u] 
to K[x 2 ,y 2 ,Z 2 ,v] induces an isomorphism between algebras of residue 
classes K[xi,yi,Zi,u]/(p) and K[x 2 ,y 2 ,Z 2 ,v]/{q): <p: x\ -> x 2 \ yi -> 
x 2 v 2 + 2 z 2 v + x 2 y 2 \ z\ -> x 2 v + z 2 \ u-t x 2 v 3 + Zz 2 v 2 + 3x 2 y 2 v + y 2 z 2 . 

It is easy to check that p induces a homomorphism between 
K[xi,yi,zi,u]/{p) and K[x 2 ,y 2 , z 2 ,v\/(q). 

We are going to show that p is onto . We already have X 2 (mod (q)) 
in the image; a straightforward computation shows that p{yiZ\) — X 2 • 
p(u ) = -2x 2 y2V + 2 z 2 v = 2 v(z 2 - x\y 2 ) — 2v (mod (q)). Thus, we have v 
(mod (q)) and therefore also Z 2 (mod (q)) in the image. Now inspection of 
p{yi) shows that we have X 2 P 2 (mod (q)) and hence also x^y^ (mod (q)) 
in the image. From p(u) we now see that we have y 2 Z 2 (mod (q)) and 
hence also y 2 Z 2 (mod (q)) in the image. This finally gives y 2 Z 2 — x\y\ = t /2 
(mod (q)) in the image. 
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Thus, p is onto. If ip were not one-to-one, then the algebra 
K[x 2 , H 2 ,Z 2 , v]/{q) would be isomorphic to an algebra K\x\,y\,z\,v\j J , 
where J is an ideal that properly contains (p). This isomorphism then im- 
plies that J can be generated (as an ideal) by a single polynomial. Since p 
is irreducible, this generating polynomial has to be p. 

Therefore, ip is one-to-one and hence an isomorphism. 

More generally, by results of Danielewski [88] and Fieseler [130], the 
hypersurface D(k) = {xy k + z 2 + 1 = 0} is not isomorphic to D(m) = 
{xy m + z 2 + 1 = 0} in C 3 if k ^ m, fc, m > 1, whereas the cylinders 
D(k) x C and D(m) x C are isomorphic subvarieties of C 4 . 

Furthermore, a combination of Danielewski’s example with our charac- 
terization of isomorphic varieties [364, Corollary 1.2] yields an example of 
equivalent cylinders with inequivalent (even nonisomorphic!) bases; those 
cylinders, however, are not hypersurfaces: 

Proposition 6.2.2. As in Danielewski’s example, letp(x,y,z) = xy — z 2 + 
1; q(x , y, z) = x 2 y — z 2 + 1. Then the varieties V\ = V ( p(x , y , z),t, u , v, w) 
and V 2 = V(q(x,y, z),t,u,v,w) in C 7 are not isomorphic, whereas the 
cylinders Vi x C and V 2 x C are equivalent in C 8 . 

Proof. First of all, the varieties V\ = V(p(x,y,z),t,u,v,w) and V 2 = 
V(q(x,y,z),t,u,v,w) in C 7 are not isomorphic since if they were, then 
V(p(x,y,z)) and V(q(x,y,z)) would be isomorphic in C 3 which is known 
not to be the case. 

On the other hand, we know that the algebras of residue classes 
C [x,y,z,t]/(p) and C [x,y, z,t]/(q) are isomorphic. This implies, by [364, 
Corollary 1.2], that the varieties V(p(xi,yi, zi),u,X 2 ,y 2 , Z 2 ,v) and 
V (q(xi,yi,zi),u, X 2 ,y 2 ,% 2 , v) are equivalent under an automorphism of C 8 . 
We note that the latter equivalence also follows from a result of Asanuma 
[28]. ■ 

Having in mind Danielewski’s example and motivated by Theorem 6.0.5, 
we ask: 

Problem 6.2.3. Let p = p(x,y,z) = xy + z 2 . Is it true that every poly- 
nomial in K[x,y,z] which is stably equivalent to p is, in fact, equivalent 
to pi 

Another natural question is whether or not Danielewski’s surfaces/cylin- 
ders have unique embeddings in C 4 . We show here that all but one of them 
do not: 

Proposition 6.2.4. For any m > 2, the hypersurface D(m) x C k ~ 3 = 
{xy m + z 2 + 1 = 0} has at least 2 inequivalent embeddings in C k for any 
k> 3. 
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Proof. We give a proof here for m = 2, just to simplify the notation. We 
get the following chain of “elementary” isomorphisms: 

(x, y , z | xy 2 + z 2 + 1 = 0) 

= (applying the automorphism <p: x — ► x, y — » y H- 1, z —> z) 

(x,y,z \ x = —xy 2 — 2 xy — z 2 — 1) 

= (x, y, 2 :, u | u = xy, x = —uy — 2u — z 2 — 1) 

= (x, y, z, u | u = — uy 2 — 2uy — z 2 y — y, x = —uy — 2u — z 2 — 1) 

= (y, z, tt | u = -uy 2 - 2uy - z 2 y - y) 

= (x, y, z | x = - xy 2 - 2xy - z 2 y - y) 

= (applying the automorphism </?:x— »x, y -» y — 1, z —> z) 

(x, y, z I xy 2 + z 2 y - z 2 + y - 1 = 0). 

Now let p = p(x,y, z) = xy 2 +z 2 + 1, q = y(x,y, z) = xy 2 +z 2 y— z 2 +y— 1. 
We are going to show that the gradients grad(y) and grad(y) have different 
numbers of zeros. This obviously implies that p and q are inequivalent under 
any automorphism of K[x,y, z] (in fact, this implies that p and q are even 
stably inequivalent). 

Compute: 

grad(p) = (y 2 ,2xy,2z) 

grad(g) = (y 2 , 2xy + z 2 + 1, 2 yz - 2 z). 

We see that grad(p) has infinitely many zeros (y = z = 0, x arbitrary), 
whereas grad(y) has no zeros. This completes the proof. ■ 

We note that for k = 3, this was also proved in [137] (by an altogether 
different method). For m = 1, the question is open: 

Problem 6.2.5. Does the hypersurface D( 1) xC = {xy + z 2 + 1 = 0} 
have a unique embedding in C 4 ? 

It is also unknown whether D( 1) has a unique embedding in C 3 (cf. [137, 
Question 1]). 

We note that Problems 6.2.3 and 6.2.5 cannot both have positive answers. 
Indeed, if the answer to Problem 6.2.5 was positive, then, since we know 
that D( 1) x C is isomorphic to D(m) x C for any m > 1, we would have 
D( 1) xC equivalent to D(m) xC in C 4 . Then, if the answer to Problem 6.2.3 
was positive, this would imply that D( 1) is equivalent to D(m) in C 3 , which 
is known not to be the case. 

This simple trick also works in a more general situation, namely: 

Proposition 6.2.6. Let V(p), p = p(x i, . . . ,x n ) be a hypersurface in C n . 
Suppose that the following two conditions hold: 

(i) V(p) x C has a unique embedding in C n+1 . 
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(ii) IfV(p) is stably equivalent to V(q) for some q = q(x i, . . . ,x n ), then 
V(p) is equivalent to V(q). 

Then, whenever V(p) x C is isomorphic to V (q) x C, it follows that V (p) 
and V (q) are isomorphic subvarieties of C n . 

The proof is obvious; we omit the details. Equally obvious is the 
following: 

Proposition 6.2.7. Let V(p), p = p(x i, . . . , x n ) be a hypersurface in C n . 
Suppose that the following two conditions hold: 

(i) V (p) has a unique embedding in C 71 . 

(ii) IfV(p ) x C s and V (q) x C s are isomorphic subvarieties of C n+S for 
some s >1, then V(p) and V(q) are isomorphic subvarieties ofC n . 

Then, whenever V (p) is stably equivalent to V(q) for some q = q(x i, . . . , x n ), 
one has V (p) equivalent to V ( q ) . 

Finally, we make one more observation inspired by Danielewski’s 
example: 

Proposition 6.2.8. Let y,x i, . . . ,x m , m > 1, be variables, and let p = 
p(x i, . . . , x m ) be an arbitrary polynomial Then hypersurfaces y • p = 1 and 
y • p k = 1 are isomorphic in C m+1 for any k > 1. 

Speaking somewhat informally, adding a polynomial q(z \, . . . , Zk) in new 
variables Zi to both polynomials in the statement of Proposition 6.2.8 can 
“spoil” this simple but delicate isomorphism (in the course of the proof, we 
shall see why), but, apparently, sometimes this isomorphism “survives” in 
higher dimensions — hence the isomorphism of cylinders. Understanding 
exactly when this happens can be the key to constructing counterexamples 
to (some of) the conjectures of this chapter. 

Proof of Proposition 6.2.8. Here we get the following chain of “elemen- 
tary” isomorphisms: 

(y,x 1 ,...,x m \y-p=l) = {y,x 1 ,...,x m \y-p=l, y 2 - p = y) 
^{y,x 1 ,...,x m ,u\yp=l, y 2 -p = y, u = y 2 } 

= (y,x 1 , . . .,x m ,u I u -p 2 = 1, u-p = y, u = u 2 p 2 ) 

= {xi, . . . ,x m ,u I u -p 2 = 1, u = u 2 p 2 ) 

= (xi,. . . ,x m ,u I u-p 2 = 1) 

- {y,xi,...,x m I y-p 2 = 1). 

Upon applying the same trick (k — 1) times, we will have y • p = 1 
isomorphic to y • p k = 1. ■ 
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Nagata’s Problem 



Let if be a field and let if [X] = if [x i, . . . , x n \ be the polynomial algebra 
in n variables xi, . . . ,x n over if. Sometimes we prefer to denote the ele- 
ments of X by x , y, z, etc. In this chapter, we study the group of if-algebra 
automorphisms Aut(if[xi, . . . ,x n ]), n>2. 

The group Aut(if[X]) has two important subgroups. The first one is 
the group A(n, if ) consisting of all affine automorphisms. The other one, 
J(n, if), is generated by all triangular automorphisms defined as fol- 
lows. Given invertible elements ai,...,a n in if and polynomials /* G 
if[x*+i, . . . , x n ], i = l,...,n ( f n is a polynomial in zero variables; i.e., 
f n £ AT), the map (aiXi + /i, a 2 x 2 + / 2 , • . • , a n x n + f n ) is called a triangu- 
lar automorphism of K[X\. The group generated by A(n, if) and J(n, if) is 
the group of tame automorphisms and is denoted by T(n, if). The automor- 
phisms of K[X\ that are not in T(n, if) are called wild. Similarly, one can 
define tame and wild automorphisms of if [X] when if is any commutative 
ring with identity. 

Finally, an automorphism a of if [rci, . . . ,# n ] is called stably tame if the 
extension of a to some K[x \, . . . ,£jv]> N > n, by a: Xi -» #*, i > n, is 
a tame automorphism of if [#i, . . . , xpj]. 

Now we state two main problems that we address in this chapter. 



Problem 7.0.1. Is it true that Aut(if [xi, . . . , x n \) = T(n, if); i.e., is every 
automorphism of if [xi, . . . , x n ] tame? 



Problem 7.0.2. Is every automorphism of if[xi, . . . ,x n ] stably tame? 
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For n = 2, Problem 7.0.1 is long known to have an affirmative answer. 
There are several known proofs of this fact originally obtained by Jung [164] 
for K = C and by Van der Kulk [194] for any field K. 

When this book was being prepared for publication, Shestakov and Umir- 
baev [342, 343] gave a negative solution to Problem 7.0.1 for n = 3 by 
proving that a particular automorphism of K[x, y, z] suggested by Nagata 
back in 1972 (see the next section) is not tame. Furthermore, they gave an 
algorithm for deciding whether or not a given automorphism of K[x,y,z\ 
is tame. 

Theorem 7.0.3. If one can effectively perform computations in the ground 
field K of characteristic 0, then there is an algorithm for deciding whether 
or not a given automorphism of K[x,y, z] is tame. 

Theorem 7.0.4 ([343]). The following automorphism v, due to Nagata, 
of a polynomial algebra K[x,y , z\ over a field K of characteristic 0 is wild: 

u(x) = x — 2 y(y 2 + xz) — z(y 2 + xz ) 2 , 
v{y) = y + z(y 2 + xz), 
u(z) = z. 

Their proof uses several ingredients, but is ultimately based on the same 
idea that underlies Whitehead’s method (see our Section 1.2). Namely, one 
has to choose a collection of “basic” automorphisms in such a way that if, 
for a given (coordinate) polynomial p, its degree can be decreased by a tame 
automorphism, then it can be, in fact, decreased by a single automorphism 
from your collection. If the latter is effectively verifiable, then you have an 
algorithm for detecting tameness. In Shestakov-Umirbaev’s proof, “basic” 
automorphisms are compositions of up to four(!) elementary or linear au- 
tomorphisms (recall that elementary automorphisms are those that change 
just one variable). This explains technical difficulty of the paper [343]. 

Later, Umirbaev and Yu [393] used a similar technique to obtain 
a stronger result: 

Theorem 7.0.5 ([393]). Two components of Nagata’s automorphism, x — 
2 y(y 2 + xz) — z(y 2 + xz) 2 and y + z(y 2 + xz), are wild coordinates; i.e., 
neither of them can be a component of a tame automorphism. 

Proofs of all these results cannot be carried on to higher dimensions, 
and there is still a possibility that the answer to Problem 7.0.1 might be 
positive for n > 4, although it seems to be a prevalent opinion that the 
answer should be negative for any n > 3. 

There are only a handful of papers on this very resistant problem, even 
though there is no lack of allegedly nontame automorphisms for any n > 3. 
The reason becomes clear after comparing this problem to Problem 7.0.2: 
it is extremely difficult to come up with invariants sensitive enough to 
distinguish between tame and stably tame automorphisms. We also note 
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that M. Smith [370] showed that the Nagata automorphism is stably tame 
and becomes tame if we extend it to an automorphism of K [x, y, z, t] by 
fixing t. 

The answer to Problem 7.0.2 is probably affirmative; at least, all known 
examples of allegedly nontame automorphisms turned out to be stably 
tame. However, the problem remains open for n > 3. There is not much 
that we can say about Problem 7.0.2 at this time, but the following recent 
result by Berson seems interesting: 

Theorem 7.0.6 ([50]). Every coordinate of K[x i, . . . , x n ] is a component 
of a tame automorphism of K[x i, . . . , xn] for some N >n. 

Even though Nagata’s automorphism is now proved to be wild, there 
is still need in new approaches to proving nontameness of particular au- 
tomorphisms since, as we have mentioned before, Shestakov-Umirbaev’s 
proof cannot be carried on to higher dimensions. In the following two sec- 
tions, we discuss two possible approaches and use Nagata’s automorphism 
as a “proving ground” since it is the simplest wild automorphism. It turns 
out that Nagata’s automorphism has some properties which distinguish it 
from known tame automorphisms. 

In Section 7.1, we consider Grobner basis techniques. There is a well- 
known Grobner basis algorithm for deciding whether an endomorphism of 
K [X] is an automorphism and, if the answer is affirmative, finding the 
inverse. An automorphism of K[X) can be reconstructed from all n 2 face 
polynomials of its inverse. In the two-variable case, we prove that just 
two face polynomials are enough, and this result is closely related to the 
fact that every automorphism of K[x, y] is tame. Examples show that a 
similar result does not hold for n > 3. We show that the reduced Grobner 
basis of three face polynomials of the Nagata automorphism obtained by 
substituting z = 0 does not produce an automorphism under any “tag” 
monomial ordering, in contrast to all examples of tame automorphisms 
of K[x,y, z] that we have tested. Based on this evidence, we formulate 
a conjecture that, if true, would imply that Nagata’s automorphism is wild. 

In Section 7.2, we try to lift the Nagata automorphism to an auto- 
morphism of the free associative algebra K(x,y,z). Obviously, every tame 
automorphism of K[X] can be lifted to an automorphism of K(X). There- 
fore, although an automorphism of K[X ] may not be tame, if it can be 
lifted to an automorphism of K(X), it may be regarded as “not wilder” 
than the corresponding automorphism of the free algebra. If an automor- 
phism of K[X ] cannot be lifted to an automorphism of K(X), we may 
ask the question of “how far” it can be lifted. The natural way to mea- 
sure the “distance” is in the language of T-ideals (i.e., ideals of K(X) that 
are invariant under all endomorphisms of K(X)). Recall that, by a result 
of M. Smith [370], Nagata’s automorphism is stably tame; i.e., it becomes 
tame if we extend it to the polynomial algebra in four variables K[x, y, z, v] 
assuming that it fixes v. Unfortunately, her procedure does not give a lift- 
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ing to Aut K(x,y, z,v) that fixes v. We have not succeeded in lifting the 
Nagata automorphism to an automorphism of K(: r, y , z), but we give an ex- 
plicit lifting to the algebra K(x, y , z)/C k , where C is the commutator ideal 
of K(x,y,z) and k is any positive integer. By a result of Umirbaev [390], 
such liftings exist for any automorphism of K[X\, but our lifting has the 
advantage that it fixes z as the Nagata automorphism itself does. 



7.1 Grobner Bases and Face Functions 

Our exposition in this section follows [106] . 

A Grobner basis is a (finite) set of special generators of an ideal of a poly- 
nomial algebra. Fixing some ordering of the monomials in K[x i, . . . ,x n ], 
the leading term of any element from the ideal is divisible by a leading 
term of some element from the Grobner basis. The basis is reduced if no 
term of its elements is divisible by the leading term of another element. 
The Grobner bases have many attractive computational properties. The 
method is widely used in computational commutative algebra and in com- 
putational algebraic geometry. The reader is referred to [13]. The theory of 
Grobner bases was used, in particular, for deciding if a given polynomial 
map F = (/i, . . . , f n ) is an automorphism. The algorithm uses “tag” vari- 
ables in a fashion related to the method for finding the relations among the 
fi s. We introduce tag variables, indeterminates j/i, . . . , y n , one for each fa. 
Let K[x i, . . . , x ni 2 /i, . . . , y n \ have a monomial ordering, where 

K[x\ , . . . j x n ] y K \yi , . . . , y n ] ? 

which means that each monomial in K\x i,...,x n ] is larger than any 
monomial in K[yi , . . . , y n \. We call it a “tag” monomial ordering. 

Proposition 7.1.1. Let F = (/i,...,/ n ) be an endomorphism of 
K[x i, . . . ,x n ]. Then F is an automorphism if and only if the ideal I of 
K[x i, . . . , x n , yi, . . . , y n \ generated by 

O/i - /i(a?i, . . . , x„), . . . , y n - fn(x 1 , . . . , x n )} 

has a generating set of the form 

{%1 ~ gi {yi 5 • • • j 2 /n)» • • • j x n ~ gn{y l) • • • ? 2 /n, ) } • 

Moreover , G := G(X) := (pi(X), . . . ,g n (X)) is the inverse of F = 
(/i, . . . ,/ n ) and the reduced Grobner basis of the ideal I with respect to 
any “tag” monomial ordering y such that K[x i, . . . , x n \ y K[yi, . . . , y n \ is 
precisely {xi - g x (y u . . . , y n ), . . . , x n - g n (yu • • • , S/n)}- 

There are several proofs of this result, by van den Essen [119] and Shan- 
non and Sweedler [335] for fields and Abhyankar and Li [9] for arbitrary 
commutative rings K. 
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It has been proved by several authors (see McKay and Wang [242], 
Li [216], van den Essen and Kwiecinski [123], and Yu [409]) that a polyno- 
mial map F = (/i, . . . , f n ) G Aut(ff [xi, . . . , x n ]) is completely determined 
by its n 2 face polynomials 

/l(0, • • • > ^n)) • • • 5 • • • ? %n— I? 0), 



«/n( 0 ) ^ 2 , • • • 5 • • • j fnfa 1 ? • • • > ^n— lj 0 )* 

Now we present a result concerning the reduced Grobner bases of the 
face functions. It is more general than that by van den Essen [119]. 

Proposition 7.1.2. Let K be any field and F = (/i,...,/ n ) G 
Aut(K[xi, . . . ,x n ]). Let 

{*^1 ^1 (Z/l 5 • • • 5 2 /n)) • • • 5 %n 9n{yii • • • > 2/n)} 

6 e reduced Grobner basis of the ideal I = (yi — /i(xi, . . . , x n ),..., y n — 
fn{x l, • • • , ^n)) K[^i, . . . , x n , 2 /i, ... , y n \ with respect to any “tag” mono- 

mial ordering >- such that K[x i,...,x n ] >- K[yi , . . . ,y n ]. Tften, /or any 
integer j = 1 , . . . , n, the set 

H {^1 — 9l{yi i • • • j 2 /n)> • • • j — 1 — — l( 2 /l » • • • j 2 /n)> ( 2/1 ? • • • j 2 /n)) 

•fy’+l — 9j + 1 (2/1 5 • • • j 2/n) • • • 5 — 2/n(2/l j • • • j 2 /n)} 

is a Grobner basis of the ideal J generated by 

fl (^T J * * * J — 1 J #7 + 1 , • • • , X n ) , • • • , 

2/n /n(^l) • • • 5 *£7 — 1? 0, Xj- f_i, . . . , X n )}. 

In general , H is not the reduced Grobner basis. The reduced Grobner 
basis is obtained after reduction of all polynomials gt(yi , . . . , y n ), i = 
1 , n, by gj(yi , . . . , y n )• ( Here j means that the corresponding 
number j does not occur in the expression.) 

Proof. The sets 

{y% ~ fi(xi,...,x n ) | i = l,...,n}, {x i -g i (y 1 ,...,y n ) | i = l,...,n} 

generate the same ideal I of K[x 1 , . . . , x n , yi, . . . , y n \. Substituting xj — 0 , 
we obtain that the set H generates in K\x 1 , . . . , Xj , . . . , x n , 2 / 1 , . . . , y n \ the 
same ideal as 

{ 2/1 fl{p^h • • • > *^7 — 1 ) 0 , • • • , X n ), • • • j 

y n /n (^ 1 ) • • • 5 Ej — li 0 , , • . . , X n )}. 

Hence these two sets generate also the same ideal in K[x 1 , . . .,x n , 2 / 1 , - - .,y n ]. 
The set if is a Grobner basis because every element from the ideal 
J is a linear combination of polynomials with leading term containing 
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x\ :z n and of polynomials divisible by Qj{yi, . . . ,j/ n )- Af- 
ter reduction of all polynomials . . . , 2/ n )? i = 1, . . . , j, . . . , n, by 

djivu • • • ,2/n)j we obtain a set H such that no monomial of an element 
from H is divisible by a leading term of another polynomial (i.e., H is the 
reduced Grobner basis). ■ 

In other words, our result states that the specialization of the Grobner 
basis works for this particular kind of ideal J, but in general the special- 
ization is not the reduced Grobner basis. In the case n — 2, we prove the 
following stronger result. 

Theorem 7.1.3. Let K be any field and let F = (fi(x,y), f 2 {x,y)) € 
Ant(K[x,y]). Then the reduced Grobner bases of its face functions produce 
automorphisms, where the Grobner bases are computed with respect to any 
(i tag” monomial ordering y such that K[x,y] y K[s,t\; i.e., 

Gbasis(s — /i(0, y),t — / 2 (f), y)) = {gi(s,t),y - g 2 (s,t)}, 
Gbasis(s-/i(x,0),t-/ 2 (x,0)) = {hi(s,t),x - h 2 (s,t)}, 

where (gi(s,t),g2(s,t)) and (hi(s,t),h2{s,t)) are elements of the group 
Aut (K[s,t\). 

Conversely, let p(t),q(t) G K[t\ such that one of them is nonconstant 
and the reduced Grobner basis G of the ideal I = ( p(t ) — x , q(t ) - y) with 
respect to an ordering y such that K[t] y K[x,y] is 

G = {fi(x,y),t- f 2 {x,y)}. 

Then (/i,/ 2 ) is an automorphism, (p(t),q(t)) are face functions of an 
automorphism (^ 1 ,^ 2 ) whose inverse is exactly (/i,/ 2 ) 7 and in this case 
K\p{t),q(t)\ = K[t\. 

Proof. Let P = {pi(x,y),p2{x,y)) be the inverse of F. Since 

(s - /i(x, y), t - f 2 (x, y)) = (x- pi(s, t),y - p 2 {s , £)), 

substituting x = 0, we obtain that {(pi(s,t),y — p2{s,t))} is a Grobner 
basis of the ideal (s - /i(0, y),t — f 2 {0, y)) in K[s, t , y) under any ordering 
that satisfies K[y\ y K[s,t\. If {pi(s,t),y—p2(s,t)} is already reduced, the 
uniqueness of the reduced Grobner basis gives that 

(9i(s,t),g 2 (s,t)) = (p 1 (s,t),p 2 (s,t)) G Aut(AT[s,t]). 

If there are further reductions, the only possibility is to reduce P2(s,t) 
modulo pi(s, t). We apply the algorithm for decomposing an automorphism 
into a product of elementary automorphisms (see Theorem 6.8.5, page 348 
in [80]). It follows from this algorithm that the reduction 



{Pi{s,t),P2(s,t)} {pi(s,t),g 2 (s,t)} 
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gives an automorphism (^1,^2) of K[s,t\. Hence the reduced Grobner basis 
of the ideal (pi(s, £), y — p 2 (s, t)) is {pi(s, £), y — g2(s , t)} and again 

(gi(s,t),g 2 (s,t)) = ( Pi(s,t),g 2 (s,t )) e Aut(.ft:[s,f]). 

Conversely, since 

1 = (pit) - x , g(t) -y) = (h(x, y),t- f 2 (x, y)), 
there exist polynomials u*, Vi G K[x, y, £], i = 1,2, such that 

/i(x,y) = (p(t) - x)u!(x,y,t) + (q{t) -y)u 2 (x,y,t), 
t - f 2 (x, y) = (p(t) - x)v!(x, y, t) + (y(£) - y)v 2 (x, y, t). 

Substituting x = p(£), y = q(t ), we obtain 

fi(p{t),q{t)) = 0 , f2(p(t),q(t)) = t. 

By [13, Theorem 2.3.4], {/i(x, y)} is the Grobner basis of the ideal of K[x, y] 

I fl K[x, y] = {/(x, y) G K[x, y] | /(p(i), q(t)) = 0}. 

On the other hand, at least one of the polynomials p(t) and q(t) essentially 
depends on £, and I DK[x,y] is a principal ideal generated by an irreducible 
polynomial. Hence fi(x , y) is irreducible. By the Abhyankar-Moh theorem 
(see Theorem 6), there exists a polynomial h(x , y) G K[x, y] such that F\ = 
(fi, h + h fi) is an automorphism of K[x, y\. Let Ff 1 = (pi(x, y), qi(x, y)) 
be the inverse of F\. We have 

(Pl(fl,f2),qi(fl,f2)) = (x,y) 



and 



(pi(o, 0>9i (o, 0 ) = (p(t),q(t))- 

By Proposition 7 . 1 . 2 , {fi(x,y),t - f 2 (x,y) - h(x,y)fi(x,y)} is a (nonre- 
duced) Grobner basis of the ideal (p(t) — x, q(t) — y) under any ordering 
that satisfies K[t] >- K[x, y], and the reduced Grobner basis under the same 
ordering is {/i(x, y), t — f 2 {x , y)}. Arguing as in the first part of the proof 
(i.e., applying again the algorithm for decomposing F\ into a product of 
elementary automorphisms), we obtain that F = {fi,f 2 ) is also an auto- 
morphism with F _ 1 ( 0 , t) = ( p(t ), q(t)). ■ 

Example 7.1.4. Consider the Nagata automorphism N = (ni,ri2,n3), 
where 



m = x - 2 y(zx + y 2 ) - z(zx + y 2 ) 2 , 
n 2 = y + z{zx + y 2 ), 



ns = z. 
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Using MAPLE 6.0, we have computed the reduced Grobner bases of all 
its face functions; i.e., the reduced Grobner basis of the ideals 

I x = ( s - ni{0,y, z),t - n 2 {0,y, z),u - n 3 (0,y, z )), 

I y = (s - ni(x,0,z),t - n 2 (x,0, z),u — n 3 (x,0,z)), 

I z = {s- ni(x, y, 0 ), t - n 2 (x, y,0 ),u- n 3 (x, y, 0 )), 

with respect to the lexicographic ordering x>y>z>s>t>u. We 
obtain that 

Gbasis(/ a ,) = {y— t+ut 2 +u 2 s,z — u, —s — 2t s — 2tus+ut^+2u 2 t 2 s+u ?> s 2 } 

— {y~ u(t 2 +us)), z—u , — (s+2t(t 2 +us)— u(t 2 +us) 2 )}, 

Gbasis(/ 2/ ) = {x — s— tus— £ 3 , z— u, — t+ut 2 +u 2 s} 

= {x — (s+t(t 2 -\-us)), z— u, — (t— u(t 2 +us))}, 

Gbasis(/ 2 ) = {x — s — 2t s ,y— £,u} = {x — (s+2£ 3 ), y— t, u}. 

Hence, the Grobner basis of I x produces the inverse of the Nagata 
automorphism 

AT 1 = (x + 2 y(zx + y 2 ) - z(zx + y 2 ) 2 , y - z(zx + y 2 ), z). 

The Grobner basis of I z produces the automorphism (x + 2 y 3 , y, z), but the 
Grobner basis of I y produces the endomorphism 

(x + y(y 2 + xz), y - z{y 2 + xz), z), 

which is not an automorphism because its Jacobian matrix is not in- 
vertible. Moreover, it is also very easy to check that the Grobner bases 
above are independent of the “tag” ordering of the monomials >- such that 
K[x,y,z] y K[s,t,u}. 

The example above shows that the case n = 3 differs from the case n = 2. 
One may suggest the following problem: 

Question 7.1.5. Let K be a field and F = (/i,...,/ n ) € T(n,K). Do 
the reduced Grobner bases for any “tag” monomial ordering of its face 
functions give rise to automorphisms? 

By Theorem 7.1.3, this question has a positive answer if n = 2. If n > 3, 
the answer is negative because M. Smith [370] showed that the Nagata 
automorphism is stably tame. If we extend it to K[x, y , z, u], assuming that 
it acts identically on the new variable v, it becomes tame; i.e., (ni, 712 , ri 3 , v) 
is a tame automorphism. The computations in K[x,y,z,v] give the same 
result as in K[x,y,z], and in this way the Nagata’s automorphism gives 
a counterexample to our question for n > 3. The following example shows 
that the answer is negative also for n = 3. The automorphism we consider 
below was discovered by Freudenburg [136] and is an example of a tame 
automorphism with nonlinear base locus. There is a conjecture that all 
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such automorphisms are wild. So, although the automorphism from [136] 
is tame, it has some “wild” properties. 

Example 7.1.6. Let K be a field of characteristic zero and let L = (y, #, z), 
A = (x, y, z + x 3 — y 2 ), and B = (x,y + x 2 J z + x 3 + (3/2)xy) be elements of 
T( 3, K). We consider the automorphism F = AoLoBoL" 1 , the functional 
composition of the polynomial maps A , L, J3, and L -1 , the inverse of 
L. Then, F = (/i,/2,/3) is an element of T(3, F), where /i = x + z 2 , 
/2 = V + z 3 + (3/2)xz, and f 3 = z + x 3 -y 2 - 3xyz + (l/4)z 2 (3x 2 - 8 yz). 

We compute the reduced Grobner bases of all its face functions; i.e., the 
reduced Grobner bases of the ideals 

I x = (s-/i(0,y,z),t-/ 2 (0,y, z),u-/ 3 ( 0,2/, 2;)), 

^2/ = (s - fi(x,0,z),t - f 2 (x,0, z),u- f 3 (x,0,z)), 

^ = (s-/i(x,y,0),t-/ 2 (^,2/,0),u-/ 3 (x,2/,0)), 

with respect to the lexicographic ordering x>y>z>s>t>u. We get 
that all Grobner bases produce tame automorphisms and one of them is 
the inverse of F. 

Now consider F~ l = G = (yi,y 2 , #3), the inverse of F, which is also in 
T(3,K): 

gi = — x 6 + x — z 2 + 2 zx 3 — 2 zy 2 + 2 x 3 y 2 — y 4 , 

g 2 = l/2(—x 9 + 2 y- 3 xz + z 3 - 3 z 2 x 3 + 3 z 2 y 2 + 3 zx 6 - 6 x 3 zy 2 + 3 zy 4 
+ 3x 4 + 3 x 6 y 2 - 3 x 3 y 4 - 3 xy 2 + y 6 ), 
g 3 = z-x 3 + y 2 . 

We also compute the reduced Grobner bases of all its face functions; i.e., 
the reduced Grobner bases of the ideals 

I x = (s- ffi (0, y,z),t- 0 2 (0, 0, z),u - 0 3 ( 0, y , z)), 

I y = (s- g 1 (x, 0, z), t - g 2 (x, 0, z), u - g 3 (x, 0, z)), 
h = {s~ 01 (x, y, 0),t - g 2 (x, y, 0 ),u - g 3 (x, y, 0)), 

with respect to the same lexicographic ordering x>y>z>s>t>u. As 
a result, we obtain 

Gbasis(/ X ) = {— 2£+2y+-u 3 , —4u+4z+At 2 — 4tu 3 +u 6 , s+u 2 }, 

Gbasis (I y ) = {—s+x—u 2 ,3z — 3s 3 —4:t 2 — 2tu 3 —u 6 —3u,2t+2u 3 +3us}, 
Gbasis(/ Z ) = {-s+x-u 2 ,2y-2t-2u 3 -3us, 

4u+4s 3 +3 s 2 u 2 — At 2 — 8tu 3 — 12 tus}. 

Again, it turns out that Gbasis(/^) gives F, the inverse of G. Besides, 
Gbasis(/ X ) produces an automorphism, but Gbasis(/ y ) does not correspond 
to an automorphism. However, if we change the “tag” monomial ordering 
(e.g., consider the lexicographic ordering x>y>z>t>s>u ), the 
reduced Grobner bases of 7 X , I y , or I z produce tame automorphisms. 
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The example above indicates that the Grobner bases of the face functions 
of a tame automorphism F with respect to a “tag” monomial ordering may 
produce tame automorphisms, but with the same “tag” monomial ordering, 
the inverse F _1 may not. It also suggests modifying slightly Question 7.1.5 
for the case n = 3. 

Conjecture 7.1.7. Let F = (/i,/ 2 ,/ 3 ) € T(3, K)\ i.e., F is a tame auto- 
morphism. There exists at least one “tag” monomial ordering that satisfies 
the condition K[x, y , z] y K[s , t, u] such that the reduced Grobner bases of 
the face functions of F under this ordering produce automorphisms. 

It is clear that the positive answer to Conjecture 7.1.7 implies that the 
Nagata automorphism is not tame. We have made a lot of experiments 
with tame automorphisms of K[x y y, z ] with different lengths of the decom- 
position into a product of affine and triangular automorphisms. In all the 
cases, we have obtained confirmations of the conjecture. The computations 
above also suggest studying the following problems: 

Question 7.1.8. Let F = (/i,...,/ n ) € Ant(K[X]). Fix a “tag” 
monomial ordering. Then: 

(i) Does at least one of the reduced Grobner bases of the face functions 
of F give the inverse of the automorphism F? 

(ii) Does at least one of the reduced Grobner bases of its face functions 
give an automorphism? 

By taking any linear automorphism involving all variables in the image of 
every variable, we obtain a negative answer to Question 7.1.8 (i). Another 
nontrivial counterexample is the following automorphism from the book by 
Cohn [80]: 

c = {x + y{xy -yz),y,z + y(xy - yz)). 

We have checked that no reduced Grobner bases of its face functions pro- 
duce the inverse of C. Moreover, C is a tame automorphism; in fact, it is 
the composition of tame automorphisms LoGoL, where L = (x,y,x — z) G 
A(3, K) and G = (x + zy 2 , y, z) G J( 3, K). 

Finally, Question 7.1.8 (ii) is still an open problem for n > 2. 



7.2 Lifting the Nagata Automorphism 

Let K(X) be the free associative algebra freely generated by the set X = 
{#i, . . . , x n }. One may consider this algebra as the algebra of polynomials 
in n noncommuting variables. The notion of tame and wild automorphisms 
of K(X) is introduced as in the case of K[X]. Every automorphism of K(X) 
induces an automorphism of K[X], and every tame automorphism of K[X] 
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can be obtained in this way (i.e. , it can be lifted to an automorphism of 
K(X)). 

The result of Makar-Limanov [233] and Czerniakiewicz [86] states that 
every automorphism of K(x, y) is tame. Again, the answer is still unknown 
for n > 2. Therefore, if an automorphism of K[X] can be lifted to an auto- 
morphism oiK{X), it may be regarded “not wilder” than the corresponding 
automorphism of the free algebra. 

Problem 7.2.1. Can every automorphism p of K [X] be lifted to an au- 
tomorphism ^ of K{X)1 If (p fixes the variables #i, . . . , x m , can the lifting 
ijj be chosen in such a way that it fixes the same variables in K (X) as p 
does? Can the Nagata automorphism be lifted? 

If an automorphism p of K[X ] cannot be lifted to an automorphism of 
j K(X), we may ask the question of “how far” p can be lifted. The natural 
way to measure the “distance” is in the language of T-ideals. 

Definition 7.2.2. An ideal of K(X ) is called a T-ideal if it is invariant 
under all endomorphisms of K{X). 

T-ideals of K(X) have important properties that make them very useful 
for applications to automorphisms. 

Proposition 7.2.3. 

(i) For every T-ideal U of K(X), there exists an algebra R such that U 
coincides with the ideal T(R) of all polynomial identities in n vari- 
ables of R; i.e., f(x i, . . . ,x n ) € U if and only if f(ri , . . . ,r n ) = 0 for 
all ri, . . . ,r n 6 R. The factor algebra F(R) = K(X)/T(R) is called 
the relatively free algebra of rank n in the variety of algebras generated 
by R. 

(ii) Every automorphism ofK(X) induces an automorphism of K(X)/U . 

Problem 7.2.4. For a given automorphism p of K[X], find the T-ideals 
U with the property that p can be lifted to K(X)/U. 

For a background on Pi-algebras, see the book by Rowen [323]. The 
theorem of Amitsur that over an infinite field the ideals T(M m (K)) of 
polynomial identities of the m x m matrix algebra M m (K) are the only 
prime T-ideals and the Razmyslov-Kemer-Braun theorem [310, 174, 59] 
on the nilpotence of the radical of a finitely generated PI- algebra give that, 
over infinite fields, for every T-ideal U there exist positive integers m and 
k such that 

T(M m (K)) 2UD T k (M m (K)). 

Since our goal is to see whether an automorphism of K[X] can be 
lifted to an automorphism of K(X), it is natural to consider liftings to 
K (X) /T k only. The following statement shows that the difficult 
part of the lifting is to K(X)/T(M m (K)) and K(X)/T 2 (M m (K)) only. 
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Proposition 7.2.5. Fork> 2, every endomorphism of K(X)/T k (M m (K)) 
that induces an automorphism of K (X) /T 2 (M m (K)) is an automorphism. 

Proof. We repeat the argument from [101] for K(X)/C k 1 where C is the 
commutator ideal of K(X). Let U = T(M m (K)), A = K(X)/U k . We use 
the same symbols (e.g., X, J7, etc.) for the images of objects from K(X) 
in A. Let A = A/U 2 . If G End(A) induces an automorphism on A, 
then there exists an endomorphism ip of A such that (p o \p and %p o <p 
both induce the identity map on A. Therefore, it is sufficient to prove 
that every endomorphism 6 of A that induces the identity map on A is an 
automorphism. Let 

0(xi) = Xi + fi, /i G t/ 2 , i = l,...,n. 

Consider any element g G U p . Since fi G U 2 , it is easy to see that 6(g) — 
g G U p+1 . Hence 6 induces the identity map on the factors [7 p /[/ p+1 , 
p = 0, 1, . . . , fc — 1. Since U k = 0 in A, A is isomorphic (as a vector space) 
to 

A©£/ 2 /i7 3 © 

and 6 acts identically on all these factors. Hence 9 is invertible (i.e., an 
automorphism) . ■ 

Umirbaev has shown that every automorphism of K[X\ can be lifted to 
an automorphism of the “free metabelian” associative algebra. This is the 
algebra K(X)/C 2 , where C = T(Mi(K)) = T(K) is the commutator ideal 
of K(X). Therefore it can also be lifted to K(X)/C k for any fc > 2. If the 
answer to Problem 7.2.1 is negative, we may ask a more precise version: 

Problem 7.2.6. Can every automorphism of K[X] be lifted to 
Aut{K(X))/T(M m (K)) and Aut (K(X))/T 2 (M m (K)) for m > 1? If ip fixes 
x \ , . . . , x m , can the lifting be chosen with the same property? How far can 
the Nagata automorphism be lifted with and without fixing one of the 
variables? 

The construction of M. Smith [370] showing that the Nagata automor- 
phism is stably tame is as follows. 

A derivation d of K[X] is called locally nilpotent if for any / G K[X] 
there exists a positive integer p such that d p (f) = 0. The derivation d is 
triangular if d(xi) = f(xi+ 1, . . . , x n ), i = 1, . . . , n. For d locally nilpotent 
(and if the characteristic of K is equal to 0), the map exp d : K[X] — ^ K[X] 
defined by 

exp<9(/) = /+-^/+^/ + ..., feK[X], 

is an automorphism of K[X\. IfwE K[X] is in the kernel of 9, then A = wd 
is also a locally nilpotent derivation. 
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The Nagata automorphism is equal to exp A for the locally nilpotent 
derivation A = w(x,y, z)<9, where d is the derivation of K[x,y,z] defined 
by 



d{x) = -2y, d(y) = z, d{z) = 0, 

and w{x , y, z) = y 2 +xz is from the kernel of d. We extend the action of d to 
K[x,y,z,v] by d(v) = 0. Then the automorphism exp(w<9) of K[x, y, z,v] 
acts identically on v and is tame because 

exp (wd) = r _1 o exp _1 (t?9) oro exp(vd), 

where r is the tame automorphism fixing x, y, z and r(v) = v + w(x, y, z). 
Besides, exp(ud) is also tame and acts on x, y, z, v by 

exp(^9)(x) = x — 2 vy — v 2 z , exp(ud)(y) = y + vz, 
exp (td)(z) = z, exp(vd)(v) = v. 

Considering the same automorphisms of K{x, y, z, v), we see that exp( , y9) is 
no longer an automorphism because v does not commute with the variables. 
But we replace exp(vd) with the tame automorphism 9 of K(x,y,z,v) 
defined by 

0(x) — x - 2 vy - v 2 z , 6(y) =y + vz, 6(z) = z, 9{v) — v. 

Considering the automorphism r = (x,y,z,v + w) of K(x,y,z,v) with 
w = y 2j rxz , we obtain the automorphism ^ = t“ 1 o0 -1 oto 0, which induces 
an automorphism (p of K[x, y, z, v] acting as the Nagata automorphism on 

x, y, z and p fixing v. Unfortunately, we are not able to restrict ^ to an 
automorphism of if(x,y, z) because the variable v is not fixed by -0 and 
the images of x and y involve v. For example, if [a, b] = ab — ba denotes the 
commutator of a and 6, then 

^(v) = v + (v — w)[y,z] + [v — w,y]z — (v — w)[v — w,z]z. 

Our problem would be solved if we were able to find an automorphism p of 
K(x, y, z, v) that induces the identity map on K[x, y, z, v] and acts on v as 

y. '). Then p~ l o x); will also induce the Nagata automorphism and will fix v. 
Therefore, if we factorize K(x, y, z, v) modulo the ideal generated by v, we 
will obtain a lifting of the Nagata automorphism to an automorphism of 
K(x,y,z). Hence it is worth trying to find a better lifting for the Nagata 
automorphism to Aut (K(x, y, z, v)). 

We have succeeded in giving an explicit lifting of the Nagata automor- 
phism to if(x,y, z)/C 2 that still fixes z. We hope that this lifting will be 
useful for further attempts to lift the Nagata automorphism to an auto- 
morphism of K(x, y, z). We have followed the procedure from the proof of 
the result of Umirbaev [390]. 

We work in the free metabelian algebra M(X) = K(X)/C 2 . We fix two 
sets of variables Y = {yi, . . . , y n } and Z = {zi, . . . , z n } and consider the 
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polynomial algebra K[Y U Z\. Changing a little the notation of Umirbaev, 
we define formal partial derivatives d/dxi assuming that 



^i_i ^ = 0 i*i 

C/%i dXi 

and, for a monomial u = ... Xi m G M(X), 



du 

dxi 



m 

~ } y Uh • • * Vik-i z ik+i ’ 
k=l 






9 x ik 

dxi ‘ 



The Jacobian matrix of an endomorphism (p of M(X) is 




which is a matrix with entries from K[Y U Z\. One of the main results 
of [ 390 ] is that the Jacobian matrix J(<p) is invertible (as a matrix with 
entries from K[Y U Z]) if and only if <p is an automorphism of M(X). 
Clearly, the invertibility of J((p) is equivalent to 0 7^ det(J(<p)) G K. 



Proposition 7.2.7. The following endomorphism a of K(x,y, z)/C 2 is an 
automorphism that induces the Nagata automorphism of K[x,y, z] 

a(x) = x - (y 2 + xz)y - y(y 2 + xz) - (y 2 + xz) 2 z, 

o{v) = y + {y 2 + xz)z, a(z) = z. 



Proof. For simplicity of notation, we assume that 

X = {x,y,z}, Y = {x x ,yi,zi}, Z = {x 2 ,y 2 , z 2 }. 

Clearly, the endomorphism a of M(X) induces the Nagata automorphism 
of K[X], In order to show that a is an automorphism, it is sufficient to 
calculate the determinant of its Jacobian matrix. The partial derivatives of 
cr(x), <r(y), and a(z) are 

da(x) 



dx 



= 1 - (yi + y 2 )z 2 - (y( + X!Z! + 2/2 + x 2 Z 2 )z 2 , 



da(x) 

dy 



= ~{yi + V2 ) 2 - (1 + (yi + y 2 )z 2 ){y\ + x x z x + y\ + x 2 z 2 ), 



My) 2 

da(z) _ 



Mv) 

dy 

dcr(z) 



= 0, 



1 + ( 1/1 + 1 / 2 ) 22 , 
da(z) _ x 



dx dy dz 

Hence the Jacobian determinant | J(cr)\ is equal to 

l - {yi + y 2 )z 2 - uz\ z\ 

-{yi + y 2 ) 2 - (1 + (yi + y 2 )z 2 )u 1 + (yi + y 2 )z 2 

where u = y\ + x\Z\ + y 2 + x 2 z 2 , and a is an automorphism. 



= 1 , 
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We sketch how, using the proof in [390], we have found the lifting of 
the Nagata automorphism. We hope that our considerations may be useful 
for attempts to lift other automorphisms suspected to be wild. Let v = 
(/i(X), f 2 (X), z) be the Nagata automorphism, where 

fi{X) = x- 2 (y 2 + xz)y - ( y 2 + xz) 2 z, f 2 (X) = y + (y 2 + xz)z. 

Tracing the proof in [390], we know that the ideals of K[Y U Z] generated 
by 

/i00-/i(^)> f2{Y)-f 2 (Z), Z1-Z2 and x 1 -x 2 , yi~y 2 , z x -z 2 

coincide and there is an invertible matrix R = (r^) G K[Y U Z] such that 

l fi(Y) - h[Z\ f 2 (Y) - f 2 {Z\ z x - z 2 ) = ( Xl - * 2 , Vl - y 2 , Zl - z 2 )R. 

We do not know if there is an algorithm to find the matrix i?, but 
nevertheless we have found one matrix with this property: 

/ 1 - 22(2/1 + 2/2) - z 2 u z\ 0\ 

R= \ “(j/i + J/2) 2 - (1 + z 2 (yi +y 2 ))u 1 + 22(2/1+2/2) Oj. 

\— *1 (2/1 + 2/2) — (2/1 + *i^i) 2 — xiz 2 u yl + xxizx + z^ 1/ 

The next step in the proof is to calculate the Jacobian matrix of any endo- 
morphism (f of M(X) that induces v on K[X] and to compute R — 

We start with 

V = (x - 2(y 2 + xz)y - (y 2 + xz) 2 z, y + (y 2 + xz)z, z), 

/ z 2 (y 2 - 2/1) 0 0\ 

R - J(ip) = I X1Z1 - x 2 z 2 0 Oj. 

\xi{y 2 — 2/1) 0 0 / 

Again, by the proof in [390], there exist elements wi, w 2 , W3 from the com- 
mutator ideal of M(X) such that their derivatives are exactly the columns 
of R — J{(p). In our case 

w 1 = [x, y\z + x[z, y] = xzy - yxz , w 2 = w s = 0, 

and one automorphism a of M(X) that lifts the Nagata automorphism is 

a = (ip(x) + <p(y) + w 2 , ip(z) + w 3 ). 

Starting with the automorphism <7, we can find a better lifting of the 
Nagata automorphism to an automorphism of K{x,y,z,v). Let 0*,r G 
Aut K (x, y, z, v ) , where 

6 * = (x — vy — yv — v 2 z , y + vz, z, v), 
r = (x, y,z,v - h w), w = y 2 + xz. 

Concrete calculations show that 

\j)* = r _1 o (#*) _1 or o0* 
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is also a lifting of the Nagata automorphism that fixes the variable v modulo 
the commutator ideal and 

= v + (v - w)[y , z]. 

Again, the elements ip*(x) and ift*(y) depend on v; for example, 

V(y) = y + wz + (v- w)[y , z\z. 

Although the expression for tp*(v) is simpler than that for ^(v) obtained 
above, we do not know how to fix the variable v in K(x, y , z, v). 

Effective lifting of automorphisms of K[X] to automorphisms of the free 
metabelian algebra M(X) relies on a result of Artamonov [20] on ideals 
of polynomial algebras: If the ideals (pi, . . . ,p&) and (#i, . . . , qu) of K[X] 
coincide, then there exists an invertible k x k matrix R with entries from 
K[X\ such that the following matrix equality holds: 

(Pi,---,Pfc) = 

We expect that the following stronger version of the theorem of Artamonov 
holds. 

Conjecture 7.2.8. Let {pi, . . .,p*;,x i, . . .,x m } and {qi, . . i , . . ., x m } 
generate the same ideal of the polynomial algebra K[X\. Then there exists 
an invertible matrix R 6 Mk+m(K[X]) such that 

(pi,...,p fc ,xi,...,x m )iZ= (gi,...,^,xi,...,x m ) 

and R has the block form 

* 0\ 

* EJ ’ 

where 0 is the k x m matrix with zero entries and E is the rax m identity 
matrix. 

If Conjecture 7.2.8 is true, this would imply a stronger version of the 
theorem of Umirbaev [390], which we state as another conjecture. 

Conjecture 7.2.9. Let C be the commutator ideal of K(X). Every au- 
tomorphism of K [X] fixing the variables x\ ,...,x m can be lifted to an 
automorphism of K(X)/C 2 also fixing xi, . . . , x m . 
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The Embedding Problem 



Let K[x \, . . . , x n ] be the polynomial algebra in n variables over a field K. 
Any collection of polynomials pi, . . . ,p m from K[x i, . . . , x n ] determines an 
algebraic variety {pi = 0, i = 1, . . . , m} in the affine space K n . We shall 
denote this algebraic variety by V(pi 1 . . . ,p m ). 

We say that two algebraic varieties V(pi, . . • ,p m ) and V(qi,...,qk) are 
isomorphic if the algebras of residue classes K[xi , . . . , x n ]/(pi> . . . ,p m ) and 
AT [xi, . . . , x n ]/ (gi, . . . , g*) are isomorphic. Here (pi, . . . ,p m ) denotes the 
ideal of K\x i, . . . , x n ] generated by pi, . . . ,p m . 

On the other hand, we say that two algebraic varieties (or, rather, em- 
beddings of the same algebraic variety in K n ) are equivalent if there is an 
automorphism of K n that takes one of them onto the other. If any two 
embeddings of an algebraic variety in K n are equivalent, we say that this 
algebraic variety has a unique embedding in K n . 

In this chapter, we address the following general problem: Given two 
isomorphic algebraic varieties in affine space AT n , find out whether or not 
they are equivalent under an automorphism of K n . We call this the Embed- 
ding problem. A special case of this problem is a well-known Embedding 
conjecture of Abhyankar and Sathaye: 

Problem 8.0.1 (the Embedding conjecture). Let p G K[x i, . . . ,x n ], 
n > 2. If the algebra K\x i, . . . ,x n ]/(p) is isomorphic to K[x i, . . . ,x n _i], 
then p is a coordinate; or, in geometric language: If a hypersurface V in C n 
is isomorphic to the hyperplane {x\ = 0}, then it is equivalent to it. 

The Embedding conjecture is known to be true for n = 2 due to 
Abhyankar and Moh [10] and Suzuki [380], and it remains open for n > 3. 
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Here we mostly address a more general Embedding problem and, in 
particular, give numerous examples of algebraic varieties (mostly hyper- 
surfaces) with inequivalent embeddings in K n (see Section 8.2). In fact, we 
give a rather general recipe for constructing such examples by borrowing 
the idea of Tietze transformations from combinatorial group theory (see 
Section 1.3). 

In Section 8.1 below, we consider plane curves and, in particular, give 
an algorithm for deciding whether two given parametric curves in C 2 are 
equivalent under an automorphism of C 2 . Here we borrow another idea from 
combinatorial group theory, the “peak reduction” idea due to Whitehead 
(see Section 1.2). 



8.1 Embeddings of Curves in the Plane 

Let K[x,y] be the polynomial algebra in two variables over a field K of 
characteristic 0. Here we contribute toward a classification of polynomials 
from K[x,y] by proving the following: 

Theorem 8.1.1. Letp{x,y) = ax n + by m + 'Z im+jn<mn c ij xi V j , a,b,dj 6 
K, i,j > 0 , a, b are not both zero, and q(x,y) — Ax r + By s + 
12i S +j r <rsbij xl y j > A,B,bij € K, i,j > 0, A, B are not both zero. Suppose 
that m does not divide n , n does not divide m, s does not divide r, r does 
not divide s, and max(m,n) ^ max(r, s). Then there is no automorphism 
a G Aut (K[x,y\) that takes p(x,y) to q(x,y). 

Polynomials of the form given in Theorem 8.1.1 can be considered 
“canonical models” for their automorphic images. Note that if, say, m 
divides n, then the degree of the polynomial p(x, y) in the statement of 
Theorem 8.1.1 can be reduced by applying an automorphism of the form 
(x — > x + p • y m / n ; y -» A y) with /i, A G K. However, there is no guarantee 
that the resulting polynomial will have the same form. This shows how 
subtle the situation is. 

In some special cases though, we can handle those polynomials with m 
divisible by n or vice versa. This is possible, for example, if some fiber 
of a given polynomial admits a one-variable polynomial parametrization 
x = u(t); y = v(t): 

Proposition 8.1.2. Suppose the fibers {p(x,y) = 0}, {q{x,y) = 0} of 
two polynomials p,q G C[x,y] admit one-variable polynomial parametriza- 
tions. Then one can effectively find out (even without knowing the 
parametrizations) if there is an automorphism ofC[x,y] that takes p to q. 

In particular, if some fiber of a given polynomial is an irreducible simply 
connected curve, then, by a well-known theorem of Zaidenberg and Lin 
[220], this fiber admits a one- variable polynomial parametrization. More 
precisely, they prove that (when K = C) every polynomial like that has 
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a canonical model of the form x k -y l with (fc, Z) = 1. Upon combining this 
with our method, we have the following: 

Theorem 8.1.3. Let p{x,y) G C [x,y] be a polynomial whose fiber 
{p{x,y) = 0} is an irreducible simply connected curve. Then some 
automorphism ofC[x,y] takes p(x,y) to x k —y l with (fc, l) = 1, and: 

(a) max(M) < d eg(p{x,y)); 

(b) either k or l divides deg(p(x, y)); 

(c) the Newton polygon of p(x,y) is either a triangle or a line segment; 
i.e., p(x, y) is of the form ax n + by m + J2im+jn<mn c ij xt V J > rn,n>0. 
If m does not divide n, n does not divide m , and m,n ^ 0, then 
m — k or l, and n = l or k, respectively. Otherwise, either p(x, y) 
is linear or the leading ” part ax n + by m + YUm+jn=mn c ij x%, \P * s a 
proper power of some other polynomial. 

Thus, in many situations it is possible to rule out polynomials with- 
out irreducible simply connected fibers just by inspection. In any case, by 
Proposition 8.1.2, there is an effective procedure for deciding if a given 
polynomial fiber is irreducible and simply connected. 

The next application of our method concerns embeddings of algebraic 
curves in the plane. 

Theorem 8.1.4. For any k > 2, there is an irreducible algebraic curve 
{with one place at infinity) that has at least k inequivalent embeddings in 
the plane C 2 . 

Our Theorem 8.1.4 says that there are arbitrarily (but finitely) many 
isomorphic algebraic curves in C 2 , all of which belong to different orbits 
under the action of the group Aut(C 2 ). A particular example of a curve like 
that would be y = x Po — yPiP*-P*, where po,pi, . . . ,Pk are distinct primes, 
Po >PiP2 Pk- 

Note that by a result of Abhyankar and Singh [12] an irreducible curve 
with one point at infinity cannot have infinitely many inequivalent embed- 
dings in C 2 . We do not know if, upon dropping the condition of having one 
point at infinity, one can find a polynomial fiber that has infinitely many 
inequivalent embeddings. 

We also note that the first example of an irreducible algebraic curve with 
one place at infinity that has at least two inequivalent embeddings in C 2 
was claimed in [11]. 

Now we say a few words about our general method. It is a well-known 
result of Jung and van der Kulk that every automorphism of K[x, y] is a 
product of elementary and linear automorphisms. The main difficulty in 
finding a canonical model for a given polynomial is to prove that one can 
find a sequence of elementary and linear automorphisms that would reduce 
the degree at every step until it is further irreducible by any elementary 
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automorphism. Then this last polynomial, whose degree is irreducible, will 
be a canonical model. 

To arrange that, we use two principal ideas. First, we mimic elementary 
automorphisms of K[x,y] by “elementary transformations” of K[t] x K[t\. 
Second, we use Whitehead’s idea of “peak reduction” (see Section 1.2) to 
arrange a sequence of elementary transformations of K[t] x K[t] so that 
the maximum degree would decrease at every step. This is described in 
Section 8.1.1 below. 

While the “peak reduction” always works for elementary transforma- 
tions of K[t\ x K[t], the first part (mimicking elementary automorphisms 
of K[x,y] by elementary transformations of K[t\ x K[t]) is where the dif- 
ficulty is. We managed to do that for polynomials of the form given in 
Theorem 8.1.1 and also for polynomials p(x,y) whose fiber {p(x,y) = 0} 
admits a one- variable polynomial parametrization x = u(t ); y = v(t) (i.e., 
this fiber is a rational curve with one place at infinity). The latter is used in 
proving Proposition 8.1.2 and Theorem 8.1.3. Those parametrizable fibers 
actually constitute the most tractable class of plane algebraic curves. It 
seemed plausible that every parametric curve would have a unique embed- 
ding in C 2 . However, the following example from [339] shows that this is 
actually not the case. 

Example 8.1.5. Let C\ be the curve given parametrically by x(t) = £ + £ 4 , 
y(t) = t 3 1 and C 2 the curve given by u(t) = t + £ 4 , v(t) = 3 t 3 + £ 6 . Then 
Ci is isomorphic to C 2 because K[t + £ 4 , t 3 ] = K[t + £ 4 , 3 1 3 + t 6 ]. Note that 
i 3 = IK*) 2 -4«(t) 3 + §«(*)). 

However, the two curves are inequivalent. The curve C\ can be presented 
as p(x,y) = x 3 — y( 1 + y) 3 = 0, whereas C 2 has a presentation q{x,y) = 
16# 6 — 8 x 3 y 2 — 12 x 3 y + y A — 51x 3 + 3 y 3 + 15 y 2 + 9y = 0. 

It is easy to see that the degree of either of these two polynomials can- 
not be reduced by a single elementary automorphism. Therefore, C\ is 
inequivalent to C 2 by a result of Wightwick [401]. 



8.1.1 Elementary automorphisms and peak reduction 

It is a well-known result of Jung and van der Kulk that every automor- 
phism of K[x, y] is a product of elementary and linear automorphisms. We 
give here a somewhat more precise statement that can be found in [80, 
Theorem 6.8.5]: 

Proposition 8.1.6. Every automorphism of K[x,y\ is a product of linear 
automorphisms and automorphisms of the form x -» x-\-f(y); y — » y. More 
precisely , if (^ 1 ,^ 2 ) is an automorphism of K[x,y] such that deg(pi) > 
deg(<?2), sa y> then either (^ 1 ,^ 2 ) is a linear automorphism or there exists 
a unique p, G K * and a positive integer d such that deg(gi — pg^) < deg(#i). 
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Now we are going to consider the direct product K[t] x K[t] of two copies 
of the one- variable polynomial algebra over K and introduce the following 
elementary transformations (ET) that can be applied to elements of this 
algebra: 

(ET1) (it, v) — )• (u + p • v k , v) for some p G K*\ k > 2. 

(ET2) (it, v) -» (it, v + p • u k ). 

(ET3) a nondegenerate linear transformation (u,v) — > {a\U + a 2 v , &iit + 
b 2 v)\ ai,a 2 M,b 2 G X. 

One might notice that some of these transformations are redundant; e.g., 
(ET1) is a composition of the other ones. The reason behind that will be 
clear a little later. 

We shall need the following: 

Proposition 8.1.7. For any pair (u,v) G K[t\ x K[t], there is a ( perhaps 
empty) sequence of elementary transformations that takes (it, v) to some 
(it, i)) such that: 

(i) the maximum of the degrees of polynomials decreases at every step in 
this sequence; 

(ii) the maximum of the degrees in (it, v) is irreducible by any sequence 
of elementary transformations. 

Comment to (i): if it happens that u and v have the same leading 
terms , then, perhaps by somewhat abusing the language, we say that the 
transformation (it, v) — >• (it — v, v) reduces the maximum of the degrees. 

Proof. We shall use the “peak reduction” method to prove this statement. 
This means the following. If at some point of a sequence of ET the maximum 
degree goes up (or remains unchanged) before eventually going down, then 
there must be a pair of subsequent ET in our sequence (a “peak” ) such that 
one of them increases the maximum degree (or leaves it unchanged) and 
then the other one decreases it. We are going to show that such a peak can 
always be reduced. In other words, if the maximum degree can be decreased 
by a sequence of ET, then it can also be decreased by a single ET. To prove 
that, we have to consider many different cases, but all of them are quite 
simple. 

Let (u,v) be a pair of polynomials from K[t] x K[t\ with, say, deg(u) < 
deg(v), and let a\ and a 2 be two subsequent ET applied to (u,v), as 
described in the previous paragraph. Consider several cases: 

(1) ai : (it, v) — > (u + p • v k ,v) for some p G K*\ k>2. 

This strictly increases the maximum degree since deg(it) < deg(u) 
by the assumption. Now we have two possibilities for a 2 since a linear 
ET cannot decrease the maximum degree. 
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(a) 0L2 • (xx + /x • v k , v) — >• (xx + fi • v k , v + A (xx + p • v k ) m ) for some 
A G AT*; ra > 2 . But this obviously increases the maximum 
degree, contrary to our assumption. 

(b) c*2 : (u + /i • v k , v) (u + fi • v k + A • v m , x;). If this (*2 decreases 
the maximum degree, then we should have /x • v k = — A • x; m , in 
which case a.2 — ckJ" 1 , and the peak reduction is just cancelling 
out ai and 0:2- 

( 2 ) ai : (xx, v) — (xx, v + /x • xx fe ) for some /x G if*; k > 2 . 

If this a\ increases the maximum degree, this can only happen when 
deg(v+ (i-u k ) = deg(xx fc ), in which case we argue exactly as in case (1). 
However, since deg(tx) < deg(v), it might happen that this a\ does 
not change the maximum degree. Then we consider two possibilities 
for OL 2 ‘- 

(a) a.2 : (w, v + /x • u k ) -> (ix, v + /x • u k + A • u m ). If this 0^2 decreases 
the maximum degree, then we should have m > k. If m = fc, 
then aia2 is equal to a single ET. If m > fc, then, in order for (*2 
to decrease the maximum degree, we must have deg(x;) divisible 
by deg(xx), in which case 0.2 alone would decrease the maximum 
degree of (u,v)\ i.e., we can get rid of oq. 

(b) 0:2 : (xx, v + ji • u k ) -» (u + A(v + /x • xx fc ) m , v + /x • xx fc ). But this a 2 
can only change the degree of the first polynomial in the pair, 
and this is not where the maximum degree was. 

( 3 ) ai is linear (i.e., : (xx, x;) —> (aiu+a2V, biXx+62^); ^1,^2, 61, 62 G K). 

Again, we have two possibilities for <22. 

(a) Q2i (aixx+a2x;,&ixx+62^) («ixx+a2x;,6ixx+62^+/x(aixx+a2x;) fc ). 
If k = 1, then a;2 is linear and therefore does not change the 
maximum degree. If k > 1, then a.2 might decrease the maximum 
degree, but this can only happen if a2 = 0, in which case we 
could decrease the maximum degree of (xx, x;) by a single ET of 
the type (ET 2 ). 

(b) The case where a.2 is of the type (ET 1 ) is completely similar. 

Thus, in any of the cases considered, if there is a “peak”, then we can 
reduce the number of ET in the sequence. An obvious inductive argument 
completes the proof of Proposition 8 . 1 . 7 . ■ 



8.1.2 A classification of parametric curves 

To prove Theorem 8 . 1 . 1 , it is clearly sufficient to prove the following: 

Proposition 8.1.8. Let p(x,y) = ax n + by m + Z im+j n<mn c ij xi y j > 
a,b,Cij G K, a, b are not both zero. Suppose that m does not divide n , 
and n does not divide m. Then no automorphism a G Aut (K[x,y]) can 
reduce the degree ofp(x,y). 
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First, we need the following: 

Lemma 8.1.9. Let p(x,y) be a polynomial of the form ax n + by m + 
^2im+jn<mn c ij xl y j > c ij ^ K- Then applying an elementary or linear 
automorphism (3 to p(x , y) gives a polynomial of the same form except per- 
haps in the case where m divides n or n divides m, say , m = kn , and 
(3: x^x + p-y k ;y^y for some p £ K* . 

Proof. The statement is obvious for a linear automorphism, so suppose 
we have an elementary automorphism /? : x — » x + p • y k \ y — > y for some 
p £ K *; k > 2. Then 



n — 1 

f3{p(x, y)) = ax n + y n y kn + by m + J2 

i = 1 

+ U + ^ . ( 8 . 1 ) 

im+jn<mn ^ s=l ^ 

Now we have to consider three cases: 

(a) kn < m. We have to show that for every monomial x l yi in (8.1), one 
has im + jn < mn. This is not obvious only for monomials of the 
form x s y k Compute 

sm + (k(i — s) + j)n = sm + kn(i — s) + jn. (8.2) 

To see that the right-hand side of (8.2) does not exceed mn, note 
that sm + kn(i — s) < sm + m(i — s) — mi since kn < m. Therefore, 
sm + (k(i — s) + j)n < mi + nj < mn by the assumption. 

(b) kn > m. In this case, the “leading part” of (3(p(x,y)) is x n + // n i/ fcn , 
so we have to show that, for every monomial x l yi in (8.1), one has 
ikn + jn < kn 2 . Again, we only have to show that for monomials of 
the form x s y k ^~ s ^ j 



skn + ( k(i — s) + j)n — kni + jn. (8.3) 

Since mi + nj < mn, after multiplying both sides by we get 
kni + ^-j < kn 2 . Since ^ > n (recall that kn > m), this yields 
kni + jn < kn 2 . Comparing this to (8.3) completes the proof in this 
case. 

(c) kn = m. The same argument as in the previous case works here 
unless /3: x -» x + p, • y k \ y — > y for some p, £ AT*, which can cause 
cancellation of the leading y m , thereby losing control. 



Proof of Proposition 8.1.8. By way of contradiction, assume there is 
a £ Aut (K[x,y\) that takes p{x,y) to some q(x,y) of smaller degree. Put 
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into correspondence to the polynomial p(x, y ) a pair of its face polynomi- 
als (p(t, 0),p(0, t)) E K\t) x K[t\. Furthermore, put into correspondence to 
elementary (linear) automorphisms of K[x,y] elementary (linear) trans- 
formations (ET) of K[t\ x K[t\ as described in the previous section. 
Then the sequence of ET that corresponds to the automorphism a takes 
(p(t, 0),p(0,t)) to some (u,v) E K[t] x K[t\. Consider two cases: 

(1) In the sequence of elementary (linear) automorphisms of K[x,y] 
that takes p(x,y) to q(x,y), we do not have a situation described 
in the proof of Lemma 8.1.9, case (c) (in which case we have, 
in particular, (%v) = (q(t,0),q(0,t)) or (g(0, t), q(t, 0))). In this 
case, by Lemma 8.1.9 and Proposition 8.1.7, there must be an 
ET that decreases the maximum of the degrees of the polynomials 
(p(t,0),p(0,f)). However, given the hypotheses of Proposition 8.1.8, 
this is readily seen to be impossible. 

(2) Otherwise, at some point in the sequence of elementary (linear) au- 
tomorphisms of K[x,y] that takes p(x,y) to q(x,y), we must have 
a situation described in the proof of Lemma 8.1.9, case (c). In other 
words, there is an initial fragment of our sequence that ends up 
with an elementary automorphism that decreases the degree. Among 
all sequences like that, we can choose one of minimal length. For 
this sequence, we have the corresponding sequence of ET applied to 
(p(t, 0),p(0, t)). Again, the last ET in this sequence decreases the 
maximum of the degrees in a pair of polynomials, in which case, by 
Proposition 8.1.7 (or rather by its proof), the length of the sequence 
of ET can be reduced. Then the length of the corresponding sequence 
of elementary (linear) automorphisms of K[x, y] will be reduced, too, 
contrary to our choice. 

This contradiction completes the proof of Proposition 8.1.8 and thereby 
of Theorem 8.1.1. ■ 



8.1.3 Embeddings of curves in the plane 

Before we get to the proof of Proposition 8.1.2 and Theorem 8.1.3, we 
make a general remark. If a polynomial fiber {p(x, y) = 0} admits a one- 
variable polynomial parametrization x = u(t); y = v(t), where u(t),v(t) 
have zero constant terms, then, by a result of McKay and Wang [242], the 
polynomial p k (x,y), where k = [C(t) : C (u(t),v(t))\, equals the resultant 
R(x,y) = Res t(u(t) — x,v(t) - y). Moreover, they prove [242, Theorem 5] 
that the highest homogeneous part of p k (x,y) is obtained the same way 
(i.e., as a resultant) from the highest homogeneous parts of u(t) and v(t). 
This implies, in particular, that the Newton polygon of p(x, y) is either 
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a triangle or a line segment; i.e., p(x , y) is of the form 

ax n + by 171 + CijV % y?\ m,n> 0 (8.4) 

im-\-jn<mn 

(see [242, Corollary 6]). Furthermore, from the fact that p(x , y) is the mini- 
mal polynomial for u(t) and v(t), it follows that, for example, p(x + y k , y) is 
the minimal polynomial for u(t) — v k (t) and v(t). This establishes a corre- 
spondence between elementary (linear) automorphisms of K[x, y] applied to 
p(x , y) and elementary (linear) transformations (ET) of K[t] x K[t] applied 
to (u(t),v(t)). Theorem 5 of [242] implies that in any sequence of elemen- 
tary (linear) automorphisms of K[x,y] applied to p(x,y), all polynomials 
have the form (8.4), and therefore, by our Lemma 8.1.9, the corresponding 
sequence of ET applied to (u(t),v(t)) can be arranged so that it decreases 
the maximum of the degrees in a pair of polynomials at every step. 

Proof of Proposition 8.1.2. First we show that both p and q have 
“canonical models”; i.e., automorphic images whose degrees cannot be re- 
duced by any automorphisms of C[x, y\. Indeed, by the remark above, both 
polynomials are of the form (8.4). If m divides n or n divides m, then we can 
reduce the degree of the polynomial by applying an elementary automor- 
phism. This elementary automorphism can be easily found: if, say, kn = m, 
then we apply the automorphism (3: x — > x + p,-y k ; y ^ y (see (8.1)) with 

Continuing in this way, we arrive at a polynomial of the form (8.4), 
where m does not divide n, n does not divide m, and, say, m < n. The 
other polynomial can be reduced to the same form with the degrees m f and 
n', respectively. Again, we can assume that m! < n f . Now, if n f ^ n, we 
conclude (by Theorem 8.1.1) that there is no automorphism of C[x, y] that 
takes p{x,y) to q(x,y). 

If n' = n, then an automorphism taking p to q exists if and only if 
a combination of a linear automorphism with some automorphism of the 
form {x -» x; y -> y + /(#)}, where deg(/) < ra/n, can take p to q. 

To figure out if this is possible, we have to consider coefficients of the 
polynomial f(x) as indeterminates and find out if the corresponding system 
of polynomial equations in those indeterminates (together with indetermi- 
nates that are the coefficients coming from the linear automorphism) has 
a solution over C. To do that, we can apply a well-known algorithm that 
makes use of Grobner bases (see, e.g., [13]). 

This latter algorithm is pretty slow in general. However, there is one 
special case of Proposition 8.1.2 where we do not have to apply this algo- 
rithm at all. This happens when we want to find out if a given polynomial 
is coordinate (i.e., an automorphic image of x). In that case, if at some 
point we get a polynomial of the form (8.4), where m does not divide n 
and n does not divide m, then the polynomial is coordinate if and only if 
max(m, n) = 1; no further analysis is needed. ■ 




8.1. Embeddings of Curves in the Plane 117 



Proof of Theorem 8.1.3. By a result of Zaidenberg and Lin [220], some 
automorphism of C[x, y\ takes p(x , y) to x k — y l with (fc, l) = 1. The polyno- 
mial fiber {x k —y l — 0} admits a one- variable polynomial parametrization 
x — t l ] y = t k . Therefore, by the remark in the beginning of this section, the 
polynomial fiber { p(x , y) = 0} admits a parametrization x = u(t)\ y = v(t), 
and there is a sequence of ET that takes (u(t),v(t)) to (t l ,t k ) so that the 
maximum of the degrees in a pair of polynomials decreases at every step 
except perhaps several terminal steps that do not change the maximum 
degree. This immediately implies parts (a) and (c) of Theorem 8.1.3. 

Then, take the last ET in the sequence that decreases the maximum de- 
gree; i.e., after applying this ET, we get a pair of polynomials whose degrees 
are {&,&}, {fc,Z}, or {/,/}. That means the preceding pair of polynomials 
in the sequence either has degrees {km, k} or {l, ml} for some m > 2. In 
any case, either k or l divides the maximum of the degrees in the preceding 
pair of polynomials and therefore also divides the degree of the correspond- 
ing two- variable polynomial. An obvious inductive argument completes the 
proof of part (b). ■ 

Now we get to the following proof: 

Proof of Theorem 8.1.4. We have to exhibit k polynomials /i, . . . , /* 
from C[x, y] such that C[x, y]/(fi) — C[x, y]/ (/i) for every i = 1 , . . . , k, but 
none of those polynomials can be taken to another by an automorphism of 
C[x,y] (here the symbol = means “is isomorphic to”). 

A particular collection of such polynomials is as follows (it is modeled 
on the corresponding example in combinatorial group theory [241]): 

fi=y — x Po + y piP2 ” Pk ^ where po,pi , . . . ,Pk are distinct primes, 

Po > P1P2 Pk ; 

f 2 =y-( X P0 - yP^P.yi. 
f 3 =y-( X p o - yPS ' Pk )PiP2 ; 

fk=y-{x PQ -y Pk ) PlP ^ Pk -K 

We are now going to show that the corresponding algebras of residue 
classes are isomorphic. It will be technically more convenient to write those 
algebras of residue classes as “algebras with relations”; i.e., for example, 
instead of C[x,y\/(fi) we shall write (x,y \ f\ = 0). Following is the chain 
of isomorphism-preserving transformations (similar to Tietze transforma- 
tions in group theory — see Section 1.3) that establishes the isomorphism 
between (x, y | f\ = 0) and (x, y | fa = 0): 

(x,y \ y = x Po — y Pl P 2 -‘-P k ) = (x,y,z \ y = x Po — y PlP ‘ 2 -" Pk ' i) z = y Pl ) 

= (x,y,z | y = x Po — z P2 "‘ Pk ; z = y Pl ) = (x,z \ z = ( x Po - z P2 '” Pk ) Pl ) 
= (x,y | y = (x Po — y p2 ‘" pk ) pi ). 
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In a similar way, one can establish the isomorphism between (x, y \ f\ = 
0) and (x, y \ fi = 0) for every i = 2, . . . , k. 

Applying our Theorem 8.1.1 shows that none of the polynomials fi can 
be taken to /j, j ^ i, by an automorphism of C[x,y] (the restriction p 0 > 
P1P2 Pk ensures that the conditions of Theorem 8.1.1 are satisfied). 

Finally, the fact that the curve {/1 = 0} (and hence any curve {fi = 0}) 
is irreducible is obvious since fi is of the form u(x) + v(y) for nonconstant 
polynomials u, v. ■ 



8.2 Embeddings of Hypersurfaces in Affine Space 

In this section, we address two general problems: 

(I) how to find out whether or not two given algebraic varieties are 
isomorphic; 

(II) how to find out whether or not two given algebraic varieties are 
equivalent. 

Both problems have received a lot of attention over the years. For a survey 
on problem (I), we refer to [415]. The most substantial contributions to 
problem (II) are [10, 11, 380, 220] for equivalence in C 2 and [158, 166, 371] 
for equivalence in higher dimensions. In the latter three papers, it is shown 
that if the codimension of an algebraic variety V in C n is sufficiently large, 
then V has a unique embedding in C n . Examples of nonuniquely embedded 
varieties in higher dimensions are given in [90] (for n = 5) and [170] and 
[415] (for n > 3). 

In this section, we give a simple generic procedure for constructing exam- 
ples of isomorphic but inequivalent varieties in any dimension. Furthermore, 
we give a very simple yet efficient criterion for distinguishing isomorphic 
but inequivalent hyper surf aces. This criterion allows us to show that iso- 
morphic hypersurfaces that we construct are actually inequivalent even 
under any holomorphic automorphism of the ambient space C n . 

We start by addressing problem (I). Our contribution to (I) is Theo- 
rem 8.2.1 below. Before we give the statement, we introduce three types 
of isomorphism-preserving “elementary” transformations that can be ap- 
plied to an arbitrary algebra of residue classes K[x \, . . . , x n ]/i?, where 
R is an ideal of K[x i,...,x n ] and K an arbitrary ground field. These 
transformations are modeled on Tietze transformations in groups (see 
Section 1.3). 

(PI) Introducing a new variable : Replace K[xi, . . . , x n \/R by 

K[xi,...,x n ,y]/R + (y - p(x i,...,x n )), where p(x i,...,x n ) is 
an arbitrary polynomial. 
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(P2) Canceling a variable (this is the converse of (PI)): If we have an al- 
gebra of residue classes of the form K [aq, . . . , x n , y]/(pi , . . . ,p m , q), 
where q is of the form y — p(aq, . . . , x n ), and pi,...,p m € 

K[x i, . . replace this by K[x i, . . . ,x„]/(pi, . . . ,p m ). 

(P3) Renaming the variables : Replace variables {x\, . . . , x n ) by 

(xi x , . . . , Xi n ), where i \, . . . , i n are arbitrary distinct indices, 
not necessarily the integers in {1, . . . , n}. 

Then we have: 

Theorem 8.2.1. Two algebras K[x i, . . . , x n ]/ (pi, . . . ,p m ) and 

K [xi, . . . , x n \/(qi , . . . , qu) are isomorphic if and only if one can get 
from one of them to the other by a sequence of transformations (P1)-(P3). 

In particular , two algebraic varieties V(pi , . . . ,p m ) and V{q\, . . . ,qk) 
in C n are isomorphic if and only if one can get from the alge - 
bra C[*i,...,a; n ]/(pi,...,p m ) to the algebra C[x x , . . . ,x n ]/(q 1 , . . . ,q k ) by 
a sequence of transformations (P1)-(P3). 

This result alone does not solve problem (I) since it does not give a hint on 
how to choose the polynomials p(x i, . . . , x n ) in (PI). Still, the result is quite 
useful because, on the one hand, it yields invariants (necessary conditions of 
isomorphism) of algebraic varieties and, on the other hand, gives an easy, 
practical way of constructing “exotic” examples of isomorphic algebraic 
varieties, thus providing, in particular, potential examples of isomorphic 
but inequivalent varieties. 

Theorem 8.2.1 also has the following interesting corollary, which says 
that, although isomorphic algebraic varieties may not be equivalent, they 
are always stably equivalent (the precise meaning of that is clear from the 
statement below). Moreover, isomorphic varieties turn out to be stably 
equivalent under a tame automorphism. Recall that we call an automor- 
phism tame if it is a product of elementary and linear automorphisms, 
where elementary automorphisms are those that change only one variable. 

Corollary 8.2.2 (see [166, 371]). If two algebraic varieties 

V(pi, . . . ,p m ) and V(qi , . . . , qk) in C n are isomorphic, then the varieties 
V{pi,. . . ,pm,x n +i,. . . ,x 2n ) and V(qi , . . . ,g*,x n +i, . . . ,x 2n ) in C 2n are 
equivalent under a tame automorphism of C 2n . 

We now describe some invariants of isomorphic varieties that can be 
obtained based on Theorem 8.2.1. Given a variety V = V(qi , . . . ,q m ) (in 
C n ) that contains another variety V' = V{r \, . . . , r s ), consider the Jacobian 
matrix J(qi , . . . ,q m ) = {§^), 1 < i < m, 1 < j < n. Then, consider the 
image of this matrix under the natural homomorphism from C[xi, . . . , x n \ 
onto C[xi, . . . , x n ]/ (ri, . . . , r s ). We denote this image by Ar(V) and call it 
the Alexander matrix of the variety V = V(qi, . . . , q m ) with respect to the 
ideal R = (ri, . . . ,r s ). This resembles the Alexander matrix of a finitely 
presented group, which plays an important role in algebraic topology — 
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see, e.g., [62] or [85]. Continuing this parallel with the classical Alexander 
matrix, we consider elementary ideals E k , k > 0, of the Jacobian matrix 
J(q±, • • • iQm) defined as follows: 

if 0 < n — k < m, then E k is the ideal of <C[xi, . . . , x n \ generated by all 
(n - k) x (n - k) minors of the matrix J(qi , . . . , q m )\ 
if n — k > m, then E k = 0; 

if n - k < 0, then E k = C[a?i, . . . , x n \. 

Denote by E k the image of E k in C[#i, . . . , x n ]/R. Then Theorem 8.2.1 
allows us to show that E k are invariants of isomorphic algebraic varieties. 
More precisely: 

Theorem 8.2.3. Let V\ and V 2 be two algebraic varieties in C n , both 
containing a variety V(ri , . . . ,r m ). Consider Jacobian matrices J\ and J 2 
of the varieties Vi and V 2 . Let E^ and E be elementary ideals of J\ 
and J 2 , respectively , defined as above, and let R = (?T,...,r m ). Then, if 
the varieties V\ and V 2 are isomorphic, one has E^ + R = E ^ + R, after 
possibly renaming the variables in E ^ + R. 

In Section 8.2.1, we give examples of applying Theorem 8.2.3 to dis- 
tinguishing nonisomorphic algebraic varieties. Note that the condition 
E^ + R = E^ + R is always actually verifiable since checking if two 
ideals of a polynomial algebra are equal can be done by using the Grobner 
basis algorithm (see [13]). As far as renaming the variables is concerned, 
there are only finitely many ways of doing it without changing the whole 
set of variables. 

Then, we consider another application of Theorem 8.2.1 — to construct- 
ing isomorphic but inequivalent algebraic varieties. The most practical way 
of doing it is illustrated by the following 

Example 8.2.4. Let f(x,y,z) be any polynomial, and let p(x,y,z) = 
x - f(x k ,y,z ); q(x,y,z) =x- f k {x,y,z ); k > 1. Then the varieties V(p) 
and V (q) are isomorphic. 

Indeed, the algebra K[x, y, z]/(p } can be generated by u = x k , y, and z 
since in this algebra we have x = f(x k ,y,z). These new generators u,y,z 
are subject to the relation u = f k (u,y,z) (i.e., the minimal polynomial of 
u,y, and z in the algebra K[x,y,z]/{p) is u - f k (u,y,z)). Therefore, the 
varieties V(p) and V(q) are isomorphic. (In Section 8.2.2, we give a more 
formal and rigorous proof of this fact based on our Theorem 8.2.1; what 
we have just given here is a somewhat informal, “fast” version of the same 
method.) 

Now that we have a procedure for constructing nontrivial examples of 
isomorphic algebraic varieties, we need invariants of equivalent varieties. 
It appears that, in many situations, one can get away with very simple 
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invariants. From now on, we are going to concentrate on hyper surfaces] 
i.e., on varieties of the form V(p). 

It turns out that the number of zeros of the gradient grad(p) is a rather 
sharp invariant of equivalent hypersurfaces. (The fact that it is an invariant 
follows immediately from the “chain rule” for partial derivatives.) Note 
that this number is, in general, different from the number of singularities 
of a hypersurface since a point where the gradient vanishes may not belong 
to the hypersurface. 

Whereas the number of singular points is an intrinsic characteristic of 
a hypersurface (i.e., it is invariant under an isomorphism), the number of 
zeros of the gradient is not, and therefore it can be used to distinguish 
isomorphic but inequivalent hypersurfaces. This can be illustrated by the 
following simple example: 

Example 8.2.5. Let p(x , y,z) = x + y + z — xyz and q(x , y,z) = x + y 2 + 
yz — xyz. Then grad(p) has two zeros, whereas grad (q) vanishes nowhere. 
Therefore, the hypersurfaces V(p) and V (q) are inequivalent. The fact that 
they are isomorphic can be established by the same method that we used 
in Example 8.2.4. The algebra K[x,y,z]/(p) can be generated by u = xy , 
y, and z since in this algebra x = xyz — y — z. These new generators are 
subject to the relation u = ( uz — y — z)y = uyz — y 2 — yz , whence the 
isomorphism. 

This example can be generalized to the following proposition: 

Proposition 8.2.6. For any n > 3, the hypersurface x\ + x<i H \-x n — 

X\X 2 . . . x n = 0 has at least two embeddings in C n inequivalent under any 
algebraic, or even just holomorphic, automorphism of C n . 

We emphasize this particular result because it contrasts sharply with 
a result of Jelonek [160] saying that the “n-cross” X\X 2 • . . x n = 0 has 
a unique embedding in C n for any n > 3. Our Section 8.2.2 contains many 
other examples of isomorphic but inequivalent hypersurfaces. 

The question of whether or not the number of zeros of the gradient can 
possibly be useful in approaching the Embedding conjecture 8.0.1 seems 
interesting in its own right: 

Problem 8.2.7. If a hypersurface V(p) in C n is isomorphic to a coordinate 
hyperplane, is it true that grad(p) vanishes nowhere? 

If the answer to this problem is negative, that would also give an example 
of a polynomial some of whose fibers are isomorphic to V(xi) and some not. 
Indeed, suppose X 0 £ C n is a zero of the gradient of p, and let p(X o) = c. 
Then the hypersurface V(p — c) has a singular point and therefore cannot 
be isomorphic to V(x{). 

There is no polynomial like that in C[x\,X 2 \ — see [10], but for 
polynomials in more than two variables the situation is unknown. 
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We note that Kaliman [169] showed that if all fibers of a polynomial 
p = p(x i,^ 2 , xs) are isomorphic to C 2 , then p is coordinate. This condition 
was relaxed in [171] to “the general fiber of p is isomorphic to C 2 ”. 

8.2.1 Invariants of isomorphic varieties 

We start with the following proof: 

Proof of Theorem 8.2.1. Since the “if” part of the theorem is obvious, 
we proceed with the “only if” part. 

We are going to show that if the algebras K[X\=K[x i,. . .,x n ]/ (pi,. . .,p m ) 
and K[x i, . . . , x n \/(qi, . . . ,qk) are isomorphic, then we can take them both 
to the same algebra by a sequence of transformations (P1)-(P3). 

We denote the former algebra by K[X]/R and the latter by K[X]/S, 
where R and S are ideals of K[x i, . . . , x n ] generated by pi, . . . ,p m and 
respectively. Upon applying the transformation (P3), we may 
assume that the latter algebra is of the form K[Y]/S, where the set Y = 
{pi, . . . , Vn) does not overlap with X. 

Then, by repeatedly applying the transformation (PI), we can get 

n 

K[X]/R S* K[X U Y]/R + J2(Vi ~ • • • , *n)>, 

i= 1 

where Ui are polynomials constructed as follows. Since K[X]/R is isomor- 
phic to K[Y]/S , there is a mapping cp: K[X]/R -> K[Y]/S such that 
Ker(<p) = 0 + i?, and 

Pi 4- S = Ui((p(x i + i?), . . . , (p(x n + R)) + S = (f{ui(x i, . . . , x n ) + R). 

Denote the ideal ~ u i( x i> • • • > x n)) by U. Now comes the crucial 

point: we are going to show that S C R + U. 

Suppose we have an arbitrary polynomial u;(pi, . . . , y n ) G S. Then 
w (yi , . • • , y n ) = w{(yi - ui) + u u ...,{y n - u n ) + u n ) = w(u 1 ,...,u n ) 
(mod U). On the other hand, p{w(u \, . . . , u n ) + R) = w(pi, . . . , y n ) + S 
since p* + S = ip(v,i + R). Thus, w(u \ , . . . , u n ) + R G Ker(<p) = 0 + i?, and 
therefore w(yi , . . . , y n ) G R + U. 

Again, since K[X]/R is isomorphic to K\Y\/S, there is another mapping 
(the inverse of <p) i/j: K[Y]/S — > K[X]/R such that Ker(^) =0 + 5; 

Xi + R = Vi(xl>(yi + 5), . . . , 'ipiyn + S)) + R = ^(v»(j/i, • . • , y n ) + S), 
and, furthermore, ^;(< p(xi + R)) = X{ + i?, which implies 
Vi(ui, ...,u n ) = Xi (mod R). 

Then we have x { - v*(pi, . . . , y n ) = x i - ^((pi - ui) + iq,. . . , (p n - 
u n ) + u n ) = Xi — (^i j • • • , u n ) (mod U) = x l -x l (mod R) = 0 (modi?). 

Therefore, X* - ^(pi, . . . , p n ) G i? + U. 
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Thus, we have shown 



R + U = R + S + U + J2i x i ~ v i) 

1=1 

and therefore that the algebra K[X\/R can be taken to K[X U Y]/R + 
S + U + J2i=i ( x i ~ v i) by a sequence of transformations (PI), (P3). By 
the symmetry, the algebra K[Y]/S can be taken to the same algebra by 
a sequence of (PI), (P3), and this completes the proof. ■ 

Proof of Corollary 8.2.2 follows immediately from the proof of Theo- 
rem 8.2.1 since transformations of the type (PI) that we have used are 
actually tame automorphisms of K[X U Y]. ■ 

Proof of Theorem 8.2.3 basically goes along the same lines as the stan- 
dard proof of invariance of Alexander ideals — see, e.g., [85]. Namely, our 
transformations (PI) and (P2) are analogous to Tietze transformations 
applied to group presentations, so that the invariance of Ek + R can be 
established similarly. 

However, there is one subtlety here. We have to show that Ek + R does 
not depend on a particular choice of generators of the ideal (#i, . . . , g m ); 
i.e., that the ideals Ek + R are well-defined. Suppose {ui, . . . , u^} is another 
generating set of (gi, . . . , g m ), and let Uj = Y^iLi Qi ' 1 < 3 < where 

Wij are some polynomials. 

Add b zero rows to the Jacobian matrix J(qi, . . . , g m ). This does not 
change any of the Ek , which can be easily seen from the definition. Enumer- 
ate those zero rows somehow, and add to the row number j the combination 
of other rows YliLi Qi' w ij-> where by Qi we denote the row of partial deriva- 
tives of the polynomial g*. Again, this operation does not affect any of the 
Ek- Now, modulo the ideal (gi,...,g m )> the combination of rows above 
equals Y^T=i ' w ij) = d{ u j)> where by d(z) we denote the row of par- 
tial derivatives of z. By the symmetry, we can obtain exactly the same 
(m + b) x (m + b) matrix over C[^i , . . . , x n ]/(qi , . . . , q m ) from the Jacobian 
matrix J(ui , . . . , Ub) without changing any of the Ek + R. This completes 
the proof. ■ 

We now illustrate Theorem 8.2.3 by a couple of examples. 

Example 8.2.8. Let p = p(x,y) = x 2 + xy + y 3 ; V\ = V(p), and q = 
q(x,y ) = x 2 + y 3 ; V 2 = V(q). Then J(p) = grad (p) = (2x + y,x + 3 y 2 ); 
J(q) = grad(g) = (2x, 3 y 2 ). Therefore, for the matrix J(p), we have Eq ^ + 
R = (2x + y, x + 3 y 2 ,p) = (x, y) (here we choose R = (p, q)). If we rename 
the variables in (by switching x and y), we shall get the same ideal 

(z,y). 

On the other hand, for the matrix J(g), we have E ^ + R = 
(2x, 3g 2 ,p, q) = (x,y 2 ). Thus, V\ and V 2 are not isomorphic. 
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The point of the next example is to present a situation where our in- 
variants can distinguish algebraic varieties with the same topology. (We 
are appealing to an intuitively clear fact that minor perturbations of co- 
efficients of a polynomial do not change the topology of a general fiber). 
We also note that Makar-Limanov [235] has come up with another way 
to distinguish varieties with the same topology by using purely algebraic 
invariants. 

Example 8.2.9. Let p = p(x,y,z) = x + xyz + z 2 \ qk = kx + xy , k E 
C. By using our invariants, we can show that V(p,qk) and V(p,q m ) are 
nonisomorphic algebraic curves in C 3 if k ^ m. 

Let R = (p, qk). Then the ideal +R for the Jacobian matrix J(p, qk) 
has the following (reduced) Grobner basis with respect to deglex term or- 
dering (see [13]): { x , z 2 , kz+yz}. The ideal E^+R for the Jacobian matrix 
J(PiQm) has the (reduced) Grobner basis {x,z 2 ,mz + yz) with respect to 
the same term ordering. 

Since the reduced Grobner basis with respect to a particular term order- 
ing is unique, it follows that the ideal E^ + R is different from + Rif 
k^m (obviously, renaming the variables cannot change this fact), whence 
we get nonisomorphism. 



8.2.2 Inequivalent isomorphic varieties 

In this section, we give various examples of inequivalent embeddings of hy- 
persurfaces in C n . In all of these examples, we use the number of zeros of 
the gradient to establish inequivalence, as described in Section 8.2. There- 
fore, in each case, we actually establish a stronger type of inequivalence 
than just being inequivalent under any algebraic automorphism of C n . All 
examples here are those of isomorphic hypersurfaces in C n inequivalent 
under any holomorphic (i.e., complex analytic) automorphism of C n . Also, 
in all of these examples, whenever we show that a given hypersurface V (p) 
has inequivalent embeddings in C n , it will follow that V (p) has inequivalent 
embeddings in C m for any m > n as well. 

Finally, we note that, by the examples below, we try to illustrate all 
possible combinations of numbers of zeros of grad(p) and grad(g) for iso- 
morphic hypersurfaces V(p) and V(q). That is, we have examples where 
grad(p) vanishes nowhere but grad(g) has infinitely many zeros, examples 
where both grad(p) and grad (q) have finitely many but different numbers 
of zeros, and so on. 

We start with the proof of Proposition 8.2.6, but before we proceed, we 
give a rigorous proof of isomorphism claimed in Examples 8.2.4 and 8.2.5 
in Section 8.2. 

In Example 8.2.4, we start with the algebra K[x,y,z]/{p), where p = 
p(x,y,z) — x — f(x k ,y,z). We are now going to give a sequence of el- 
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ementary transformations (P1)-(P3) that will bring us to the algebra 
K[x, y , z]/ ( q ). It will be technically more convenient to write those algebras 
of residue classes as “algebras with relations”; i.e., for example, instead of 
K[x,y,z]/{p) we shall write (x,y, z | p = 0). The symbol = below means 
“is isomorphic to” . 

(x,y,z\x = f{x k , y, z)) **(x,y,z,u | a: = f(x k , y, z ), u = x k ) 

= (x,y,z,u \x = f(u, y,z), u = x k ) 

= {x,y,z,u \x = f(u, y, z), u = f k (u, y, z)) 

= ( y,z,u\u = f k (u, y, z)) = {x,y,z | ar = f k (u, y, z)). 

For Example 8.2.5, the sequence of elementary transformations looks as 
follows: 



(x,y,z \ x = xyz — y — z) = (x,y,z,u \ x = xyz — y — z, u — xy) 

= (x,y, z,u \ x = uz — y — z, u = xy) = (y,z,u \ u = ( uz — y — z)y) 
^{x,y,z\x = (xz-y- z)y) = (x,y,z\x = xyz - y 2 - yz). 

From now on, we are going to use a faster, slicker way of establishing 
isomorphism, as is done in Section 8.2. We are now ready for: 

Proof of Proposition 8.2.6. Let p = p(x i, . . . , x n ) = x\ + x 2 + • • • + 

x n — X\X 2 • • • x n . Then, since the algebra C[#i, . . . , x n ]/(p) can be gen- 
erated by X 1 X 2 • • -x n -i,X 2 , • - - ,x n , we have, as in Example 8.2.5, the 
hypersurface V(p) isomorphic to V{q), where q = x\ — {x\x n — x 2 — 

X n )x 2 X S . . . £ n _i =Xi+ xlx 3 ...X n + X 2 X% . . . X n —i ~\ h X 2 . . . X n - 

X\X 2 • • • x n . 

Now compute the gradients: grad(p) = (1 — x 2 . . . x n , 1 — x\x^ . . . x n , . . . , 
1 — xix 2 . • . x n -i). This gradient has (n — 1) zeros since, for grad(p) = 0, 
one must have x\ = x 2 — • • • = x n , and hence x™ ~ l = 1. 

On the other hand, grad(g) = (1 — x 2 . . . x n , 2x 2 Xs • • • x n + • • • + 
x 3 . ..x n -ix n - X 1 X 3 . . .x n , . . . ,x 2 . . -x n -i - x\x 2 . . .£ n -i). We are going 
to show that this gradient never vanishes. 

Indeed, from the first component, we see that none of the x 2 , . . . , x n can 
be equal to 0. Then, from the last component, we get x\ = 1. Now, if we 
divide each component except the first and the last ones by xs . . .x n -i, 
we shall get a homogeneous system of linear equations in x 2 , x 3 , . . . , x n _i 
(note that x n will cancel out since x\ — 1). The first equation is 2x 2 + £ 3 + 
• • • + x n —\ = 0, and other equations are obtained from it by repeatedly 
applying the cyclic permutation on variables. A system like that is easily 
seen to have only the trivial solution x 2 = x% = • • • = £ n -i = 0, and then 
the first component of grad(^) is not zero. ■ 

One more example that seems to be worth emphasizing is that of the 
tom Dieck-Petrie hypersurface in C n , n > 3, determined by the polynomial 
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p(x, y, z) = x 2 z — y 3 z 2 — 3y 2 z + 2x — 3y — 1 (see [97]). This hypersurface 
has several remarkable properties — see [168]. Here we prove: 

Proposition 8.2.10. Let V(p) be the hypersurface in C n , n > 3, de- 
termined by p(x, y, z) = x 2 z — y 3 z 2 — 3 y 2 z + 2x — 3y — 1. Then there is 
a hypersurface V ( q ) isomorphic to V (p) but such that there exists neither 
an algebraic nor a holomorphic automorphism of C n that takes V (p) onto 

V(q). 

We note that the part about an algebraic automorphism was established 
in [170] by using different, somewhat more complicated techniques. 

Proof of Proposition 8 . 2 . 10 . Let p = p(x, y , z) = x 2 z — y 3 z 2 — 3y 2 z + 
2x — 3 y — 1. Since in the algebra C[x, y, z]/ (p) one has x = \y 3 z 2 + | y 2 z — 
\x 2 z + §y + this algebra can be generated by x 2 ,y, z. Therefore, as in 
Example 8.2.4, the hypersurface V(p) is isomorphic to V(q), where q = 
X - {\y z z 2 + \y 2 z - \xz + § y+ |) 2 . 

Now compute the gradients: grad(p) — (2 + 2xz , —3 — 3 y 2 z 2 — Qyz , x 2 — 
2 y s z — 3 y 2 ). This gradient has infinitely many zeros since the common zero 
locus of the three components is the same as that of (x — y, xz + 1). 

On the other hand, grad(g) = (1 + z • Q, ~{\y 2 z 2 + 3 yz + |)Q, — (y 3 z + 
| y 2 - |x)Q), where Q = Q(x,y,z) = ( \y 3 z 2 + | y 2 z- \xz + \y+\). This 
gradient never vanishes. 

Indeed, the second component is — | (yz + 1 ) 2 Q. Since Q ^ 0 (otherwise, 
the first component would be 1), we get yz = —1. Now, from the last 
component, we get x = y 2 . Plug this into the first component and get 
z = 0, which contradicts yz = — 1. ■ 

To construct similar examples in C 2 is somewhat more difficult since 
there is “less room” there. However, we managed to do that as well. 

Example 8.2.11. Let p = p(x , y) = x 2 + y 2 — 1. We claim that the curve 
V(p) has at least two inequivalent embeddings in C 2 . 

Since x 2 + y 2 — {x + iy)(x — iy ), where i 2 = — 1, the algebra C[x,y]/ (p) 
is isomorphic to C[x, y]/(xy — 1). In this latter algebra, one has x = x 2 y, so 
that C[x,y]/(xy — 1) can be generated by x 2 ,y. Therefore, the curve V(p) 
is isomorphic to V(q), where q — x 2 y — 1. 

Obviously, grad(p) vanishes only at the origin, whereas grad(g) has 
infinitely many zeros. 

Example 8.2.12. Let p = p(x, y) — x — x 2 — x 2 y — 1. Since in the algebra 
C[x, y]/{p) one has x = x 2 +x 2 y + 1, this algebra can be generated by x 2 ,y. 
Therefore, the curve V(p) is isomorphic to V(q), where q = x-(l+x+xy) 2 . 

Clearly, grad(p) = (1 — 2x — 2xy, —x 2 ) vanishes nowhere, whereas 
grad(g) = (1 - 2(1 + y)( 1 + x + xy), -2x(l + x + xy) vanishes at the 
point (0, — \). 
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Example 8.2.13. Let p = p(x,y) — x — y — x 2 y — x 2 y 2 . In the algebra 
C [x,y]/(p], one has x = y + x 2 y + x 2 y 2 , and hence this algebra can be 
generated by x 2 y , y. Therefore, the curve V (p) is isomorphic to V (q), where 
q = x-(x + y + xy) 2 y. 

To find zeros of grad(p) and grad(g) is not so easy in this example, but 
constructing Grobner bases for both ideals (p x ,Py) and (q x ,q y ) facilitates 
the process. The Grobner basis (with respect to the lexicographic term 
ordering with y > x) for ( p x ,p y ) turns out to be {4 y 3 + 6 2 + 2 y + x + 
2, 4 y 4 + 8 y 3 + 4 y 2 + 2y + 1}, whereas for (q x , q y ) it is {36 y 2 + 24 y + 8x + 
27, 42/ 3 + 42/ 2 + 3y + 1}. 

Now we see that grad(p) has four zeros, whereas grad(g) has three zeros. 



8.3 The Embedding Conjecture 
for Free Associative Algebras 

In this section, we show that a problem similar to the Embedding conjecture 
admits an easy solution if a free associative algebra is considered in place 
of a polynomial algebra. 

Let A n = K(x i, . . . , x n ) be a free associative algebra over a field K. For 
u E A n , let (u) denote the ideal of A n generated by u. We are going to 
give examples of elements u E A nj n > 3, such that the factor algebra of 
A n over the ideal (u) is isomorphic to A n _ i, and yet u is not a primitive 
element of A n (i.e., it cannot be taken to x\ by an automorphism of A n ). 

Theorem 8.3.1. Let u = x\ — (x 2 + #2^3 )^3 . Then , for any n > 3, 
the factor algebra A n /(u) is isomorphic to A n -\, but u is not a primitive 
element of A n . 

This, combined with Makar-Limanov’s “Freiheitssatz” [234], gives 
a negative answer to a question of G. Bergman [80, p. 336, Problem 6]: 

Corollary 8.3.2. Let K be a field of characteristic 0. Let u = x i — (x 2 + 
X 2 Xs)xs. Then, for any n > 3, the factor algebra A n /(u) is isomorphic to 
A n -i, but no automorphism of A n takes the ideal ( u ) to (xi). 

We note that u = x\ — (x 2 + X 2 Xs)x^ is just one of the many elements 
providing a negative answer to Bergman’s question. Other examples include 
Uk,m — x\ — (x\ + X2X™)x™ for any k > 2, m > 1. 

Proof of Theorem 8.3.1. First of all, u is not primitive since the Fox 
gradient of u (see Section 9.2.1) is (1 - ^10:3, -x 2 , 0), which is not left 
unimodular. 

Now we are going to show that the factor algebra A n /(u) is isomorphic 
to A n _ 1. It will be technically more convenient to write factor algebras 
as “algebras with relations”; i.e., for example, instead of A n /(u) we shall 
write (xi, . . . , x n | u = 0). Following is the chain of isomorphism-preserving 
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transformations (similar to Tietze transformations in group theory — see 
Section 1.3) that establishes the isomorphism between (x,y,z \ x = (x 2 + 
yz)z ) and {x,y,z | x = 0): 

{x,y,z \ x- (x 2 + yz)z ) = (x,y,z,t\x = (x 2 + yz)z, t = x 2 ) 

^ (x,y,z,t\x = (t + yz)z, t = x 2 ) ^ (y,z,t \ t = {{t + yz)z) 2 ) 

^ (x,y,z\x = {(x + yz)z) 2 ) ^ (x, y, z \ x = (x + yz)z(x + yz)z) 

= (x,y,z | x - yz = xzxz) = {x, y, z \ x = {xzx + y)z) 

= {x, y, z | x = yz) = (x, y, z \ x = 0). 



Proof of Corollary 8.3.2. Let u = x\ - (x 2 + X 2 Xs)x 3 G K(x i, . . . , x n ), 
and suppose there is a primitive element p £ (u) . Apply an automorphism 
c p that takes p to x. Then x G (<^(u)). But (p(u) depends on at least two 
variables because it is not primitive and irreducible; therefore, by the Frei- 
heitssatz [234], every element of (u) depends on at least two variables, too. 
This contradiction completes the proof. ■ 

If n = 2 and the ground field has characteristic 0, the isomorphism of 
A 2 /(u) to A\ does imply that u is a primitive element of A^. 

Proposition 8.3.3. Let K be a field of characteristic 0, and let u G 
K(x,y). If the factor algebra K(x,y)/(u) is isomorphic to K(x), then u 
is a primitive element of K(x,y). 

Proof. Let u G K(x,y), and let K(x , y)/{u) be isomorphic to K(x , y)/{x). 
Let u a G K[x,y ] be the natural abelianization of u. Then K[x,y]/(u a ) is 
isomorphic to if[x, y]/(x). By Abhyankar and Moh’s theorem (Theorem 6), 
u a is a coordinate in K[x , y], i.e., it can be taken to x by an automorphism 
of K[x,y]. 

Since there is an isomorphism ip between K(x,y)/(u) and K(x,y)/(x) 1 
we see that every element of the commutator ideal of K(x, y) belongs to the 
ideal (x); otherwise, p would not be one-to-one because p(xy—yx) G (x) for 
any p. Therefore, the ideal (u) contains all elements whose abelianization is 
u a . One of them must be primitive; then, as in the proof of Corollary 8.3.2, 
we conclude that u must be primitive by the Freiheitssatz [234] . ■ 




9 

Coordinate Polynomials 



Let P n = K[x i, . . . , x n ] be the polynomial algebra over a field K of char- 
acteristic 0. Recall that a polynomial p G P n is coordinate if it can be 
included in a generating set of cardinality n of the algebra P n . 

The main question under consideration in this chapter is how to recognize 
coordinate polynomials. At the time of this writing, there is no algorithm 
known for recognizing coordinates in P n if n > 3. On the other hand, 
there are several different algorithms that recognize coordinates in P 2 ; we 
describe one of them in the next section. 

In the hope of getting at least some reasonable necessary conditions for 
a polynomial to be coordinate, we consider what we call the outer rank of 
a polynomial: 

Definition 9.0.1. The outer rank of a polynomial p G K[x 1 , . . . , x n \ is the 
minimal number of generators Xi on which an automorphic image of p can 
depend. 

It is clear that the outer rank of a coordinate polynomial equals 1. The 
converse is not true, but irreducible polynomials of outer rank 1 with zero 
constant term are obviously coordinates. 

The question of actually finding the outer rank of a given element has 
been considered for various algebraic systems. The first significant result 
in this direction should be attributed to Whitehead: based on his powerful 
combinatorial technique, it is possible to find the outer rank of an element 
of a free group — see [227]. 

Mikhalev and Zolotykh [273] have come up with an important obser- 
vation: the outer rank of an element of a free Lie algebra turns out to 
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be equal to the rank (i.e., to the minimal number of generators) of the 
right ideal of the enveloping free associative algebra generated by partial 
(noncommutative) Fox derivatives of this element. 

A similar result has been proved by Umirbaev [391] for elements of a free 
group. More specifically, the outer rank of a free group element is equal 
to the rank of the right ideal of the free group ring generated by par- 
tial Fox derivatives of this element (see our Section 1.4). This result was 
independently obtained also by Bergman [47] . 



9.1 Coordinates 

of Two- Variable Polynomial Algebras 

The results described above might tempt one to assume that also in the 
case of a polynomial algebra P n the outer rank of a polynomial p equals 
the rank of the ideal of P n generated by partial derivatives of p. This, 
however, is not the case, as the following example shows. Partial derivatives 
of the polynomial p = x\ + x\x^ generate the whole algebra P 2 as an ideal; 
therefore, they generate an ideal of rank 1 (i.e., a principal ideal). At the 
same time, it is easy to see that the outer rank of this polynomial equals 2 . 

The analysis of this example has led us to the following observation. 
In the course of constructing a Grobner basis of a given ideal, one uses 
“reductions”; i.e., transformations of the following type (see [13, pp. 39- 
43]): given a pair (p, q) of polynomials, set S(p , q) = where 

1 . t.(p) is the leading term of p (i.e., the leading monomial together with 
its coefficient); L = 1. c. m.(l. m.(p), 1. m.(g)) (here, as usual, 1. c. m. means 
the least common multiple, and 1 . m.(p) denotes the leading monomial of 
p). In this book, we’ll always consider what is called “deglex ordering” 
in [13] — where monomials are ordered first by total degree and then 
lexicographically with x\ > x<i > • • • > x n . 

Now a crucial observation is as follows. These Grobner reductions appear 
to be of two essentially different types: 

(i) regular , or elementary , transformations. These are of the form 
S(p, q) — ol * p — r - q or S(p, q) = a • q — r • p for some polynomial r 
and scalar a £ K*. This happens when the leading monomial of p is di- 
visible by the leading monomial of q (or vice versa). The reason why we 
call these transformations elementary is that they can be written in the 
form (p, q) — > (aq p, 0 : 2 * 7 ) ' Af , where M is an elementary matrix (i.e., a ma- 
trix that (possibly) differs from the identity matrix by a single element 
outside the diagonal). In the case when we have more than two polyno- 
mials (pi, . . . ,Pfc), we also can write (pi, . . . ,p k ) ->► (oipi, . . . , a k p k ) • M, 
where M is a k x k elementary matrix; elementary reduction here is ac- 
tually applied to a pair of polynomials (as usual) while the other ones are 
kept fixed. Sometimes, it is more convenient for us to get rid of the co- 
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efficients and write (pi, . . . ,Pk) -> (pi> • • • ,Pk) • M, where M belongs 
to the group GEfc(P n ) generated by all elementary and diagonal matrices 
from GLfc(P n ). It is known [379] that GE^(P n ) = GL&(P n ) if k > 3, and 
GE 2 (P n ) ^ GL 2 (P n ) if n > 2 — see [78]. 

(ii) singular transformations — these are nonregular ones. 

It appears that the reason why the outer rank of the polynomial p = 
x\ + x\x 2 above is greater than the rank of the ideal I^p) of P 2 generated 
by partial derivatives of p is the presence of singular transformations in 
the corresponding reduction process. 

We say that a polynomial p £ P n has unimodular gradient if /^( p ) = P n 
(in particular, the ideal 7^0) has rank 1 in this case). Note that if the 
ground field K is algebraically closed, this is equivalent to the gradient 
being nonvanishing. 

Then we have: 

Theorem 9.1.1. Let a polynomial p £ P 2 have unimodular gradient. Then 
the outer rank of p equals 1 if and only if one can get from (di(p) , d 2 (p)) 
to (1,0) by using only elementary transformations or, in the matrix form: 
if and only if (di(p), d 2 (p)) • M = (1, 0) for some matrix M G GE 2 (P 2 ). 

Our proof of Theorem 9.1.1 is based on a generalization of Wright’s Weak 
Jacobian Theorem [404] — see Proposition 9.1.9. 

Remark 9.1.2. Elementary transformations that reduce (di(p) , d 2 (p)) to 
(1,0) can actually be chosen to be Grobner reductions (i.e., to decrease 
the maximum degree of monomials at every step) — we prove it based on 
a recent result of Park (see Proposition 9.1.10). 

Now we show how one can apply this result to the study of coordinate 
polynomials. It is easy to show that a coordinate polynomial has a uni- 
modular gradient, but the converse is not true. On the other hand, we 
have: 

Proposition 9.1.3. A polynomial p £ P n is coordinate if and only if it 
has outer rank 1 and a unimodular gradient. 

Combining this proposition with Theorem 9.1.1 yields the following: 

Theorem 9.1.4. A polynomial p £ P 2 is coordinate if and only if one 
can get from (d\(p), d 2 {p)) to (1,0) by using only elementary Grobner 
reductions. 

This immediately yields an algorithm for detecting coordinate polyno- 
mials in P 2 . Our algorithm is very simple and fast: it has quadratic growth 
with respect to the degree of a polynomial. When p is revealed to be a co- 
ordinate polynomial, the algorithm also gives a polynomial that completes 
p to a basis of P 2 . 
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In the case where K = C, an alternative, somewhat more complicated 
algorithm has been reported in [120]. It is not known whether or not there 
is an algorithm for detecting coordinate polynomials in P n if n > 3. 

Remark 9.1.5. For n > 3, Theorem 9.1.1 is no longer valid since in this 
case, by a result of Suslin [379], the group GL n (P n ) = GE n (P n ) acts tran- 
sitively on the set of all unimodular polynomial vectors of dimension n, yet 
there are polynomials with unimodular gradient but of the outer rank 2 — 
see the example above. The “only if” part, however, is valid for an arbitrary 
n > 2 — see Proposition 9.1.6. It is also easy to show that one always has 
orankp > mxik(I d ^ p )) — see Lemma 9.1.8. 

Finally, we show that our method also yields an algorithm that, given 
a coordinate polynomial p E P 2 , finds a sequence of elementary automor- 
phisms (i.e., automorphisms of the form x\ — > #i + /(# 2 ); x 2 -» X 2 together 
with linear automorphisms) that reduces p to x\ — see Remark 9.1.11. 

9.1.1 Proofs of main results 

We start by fixing some notation. We write orank(p) for the outer rank 
of a polynomial p E P n , and I d ^ for the ideal of P n generated by partial 
derivatives di(p ), . . . , d n (p) of p. Naturally, (p) denotes the partial deriva- 

tive of p with respect to Xk- For an automorphism (p that takes Xi to pi , 
1 < i < n, the Jacobian matrix is defined as follows: J \p = {dj(pi))i<ij< n . 

We need the “chain rule”: if p = p(q) for some endomorphism <£>, then 
(in the matrix form) 

(di(p ), . . . , d n (p)) = {ip{di(q)), ip{d n {q))) ■ J v , (9.1) 

where J ^ is the Jacobian matrix of p. There is also a useful product rule 
for the Jacobian matrices: 



J<p(lp) * 1 7 - 0 . ( 9 * 2 ) 

Proof of Proposition 9.1.3. 

(i) Suppose p is coordinate. Then orank(p) = 1 since, by definition, there 
is an automorphism of P n that takes p to x\. 

Now let p = p(x\) for some automorphism p. Then, applying the “chain 
rule” (9.1), we get 

(di(p), . . . , d n (jp )) = (<^(di(a;i)), . . . , ip(d n (x i))) • J v . 

This gives 

(d 1 (p),...,d B (p)) = (l,0 > ...,0)- J v . (9.3) 

Since < p is an automorphism, the Jacobian matrix J v is invertible, so that 
(9.3) gives 

(d 1 {p),...,d n {p))-j; 1 = (i, 0,...,0), 

implying 1 € Id(p), so I d ( p ) = P„. 
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(ii) Now suppose orank(p) = 1 and I d ( p ) = P n • After applying an auto- 
morphism (if necessary), we can reduce p to the form p = ^ aix\, where 
oti G K. Therefore, d\(p) should generate the whole P n as an ideal. But 
this is only possible if oti = 0 for i > 1. Thus, p is coordinate. ■ 

Our next goal is to prove Theorem 9.1.1. First we prove the “only if” 
part of this theorem in a more general form: 

Proposition 9.1.6. Let p € P n be a coordinate polynomial. Then one 
can get from (di(p), . . . , d n (p)) to (1,0,..., 0) by using only elementary 
transformations. 

We need one lemma: 

Lemma 9.1.7. If p is an automorphism of P n , then the Jacobian matrix 
J<p belongs to GE n (P n ). 

Proof. The fact that belongs to GL n (P n ) follows from the “chain rule” . 
Now there are two cases: 

(i) n > 3. Then we just have GE n (P n ) = GL n (P n ) by a result of [379]. 

(ii) n = 2. In this case, the automorphism group of P 2 is generated by 
linear automorphisms together with automorphisms of the form x\ — > X\ + 
f(x 2 ); X 2 — )■ X 2 , where polynomial f(x 2 ) does not depend on x\ (see [164]). 
It is easy to see that the Jacobian matrix of any of those automorphisms 
belongs to GE 2 (P 2 ); hence the Jacobian matrix of any automorphism of P 2 
belongs to GE 2 (P 2 ) as well — this follows from the product rule (9.2). 



Now we are ready for the following proof: 

Proof of Proposition 9.1.6. First of all, I d ( p ) = P n by Proposition 9.1.3. 
Since p is coordinate, we have p = p{x{) for some automorphism (p. Arguing 
as in the proof of Proposition 9.1.3, we get 

{di(p),...,dn(p))- J~ l = (1,0,...,0). (9.4) 

Since J~ l G GE n (P n ) by Lemma 9.1.7, (9.4) implies that one can get from 
(di(p), . . . , d n (p)) to (1,0,..., 0) by using only elementary transformations. 



Proof of Theorem 9.1.1. The “only if” part follows from Proposition 
9.1.6, so we proceed with the “if” part. 

Again, we need one general lemma: 

Lemma 9.1.8. For an arbitrary polynomial p G P n , one has orank (p) > 
rank(/ d (p)). 

Proof. Let orank(p) = k. Then, upon applying an automorphism (if nec- 
essary), we can assume that p has the form p = p(x 1 , . . . ,#&). Note that 
the rank of I d ( p ) does not change under applying an automorphism to p — 
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this is clear from the “chain rule” (9.1). Now we see that the ideal I d ( p ) can 
be generated by k elements. ■ 

Back to the proof of Theorem 9.1.1, we see that by Lemma 9.1.8 the 
only case where we could possibly have orank(p) ^ rank(/^( p )) is where 
orank(p) = 2, whereas rank {Id( P )) — 1- 

Since one can get from (di(p), d 2 (p)) to (1,0) by using only elementary 
transformations, we have 

{di(p),d 2 (p)) ■ M = {1,0) 

for some matrix M E GE 2 (P 2 ). 

Let M = (qlrl)y so that 

di(p)ri + d 2 (p)r 2 = 0. (9.5) 

Since M E GL 2 (P 2 ), the column (£*) must be unimodular; i.e., for some 
polynomials si, s 2 E P 2 , we have 

s\ri + s 2 r 2 = 1. 

This together with (9.5) gives 

( d t *“)■«-(: 

The matrix Mi belongs to GE 2 (P 2 ) because it is triangular. Hence the 
matrix J = ( dl s ^ d2 s ^) belongs to GE 2 (P 2 ), too. 

Now we arrive at the most crucial point of the proof. We are going to 
use a result of [404] (Weak Jacobian Theorem) in a somewhat stronger 
form. The result itself says: if p, q are two polynomials from P 2 , and the 
corresponding Jacobian matrix J = ^(g) ) belongs to GE 2 (P 2 ), then 

p and q generate P 2 ; in particular, they both are coordinate polynomials. 

A closer look at the (inductive) argument in the proof of this result (see 
[404, p. 250]) shows that the condition on the second row of the matrix 
J to have the form (di(q) d 2 (g)) is not actually used. In other words, the 
result can be strengthened as follows: 

Proposition 9.1.9. If J is a matrix of the form ( dl s ^ d2 s ^)> an d J G 
GE 2 (P 2 ), then p is a coordinate polynomial. 

Proof of this proposition repeats the proof of Theorem 6 from [404] un- 
til “the very last moment” , when the actual induction step is being done. 
Since we do not know a priori whether our matrix J is the Jacobian ma- 
trix of some endomorphism <p, we cannot talk about composing with 
the elementary automorphism ^ constructed in the course of the proof of 
Theorem 6 in [404, p. 250]. 

Instead, having in mind the product rule (9.2), we just consider the 
matrix J* = 'if(J) • J^, where J ^ is the Jacobian matrix of and hence an 
elementary matrix. The same expansion of J* as the one of in [404, 
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p. 250] shows that the “normal form” of J* has smaller length than that 
of J. Hence we may refer to the inductive assumption as soon as we show 
that the first row of the matrix J* still has the form ( di(q) d 2 (q) ) for some 
polynomial q G P2- 

Indeed, it is straightforward to see that the first row of the matrix J* 
is ( d 1 ('ip(p)) d 2 ('ip(p)) ). This completes the proof of Proposition 9.1.9 and 
thereby the proof of Theorem 9.1.1 since we have shown that p is a co- 
ordinate polynomial, in particular orank(p) = 1. ■ 



Proof of Theorem 9.1.4. Our proof is based on the following result of 
H. Park [298] (we give it here in a little stronger form): 

Proposition 9.1.10. Let A = (£ f ) G GE2(P2) and l.m.(A) = 
(l m (r) l m (5) ) ^ e ma ^ x of its leading monomials. Then , either at 

least three of the entries of A are constants or one of the rows of 1. m.(A) is 
a monomial multiple of the other row and one of the columns is a monomial 
multiple of the other column. 

We give here a brief exposition of the proof. 

The proof is by induction on the (minimum) number of elementary 
matrices Ei in a decomposition of the form 

A = D • Ei . . . Ek, k > 2, 

where D is a diagonal matrix and E \ ... Ek elementary matrices. 

Since the matrix A is invertible, leading terms should cancel out when 
we compute the determinant of A ; this means in the matrix 1. m.(A) either 
all of the entries are constants, one of the entries is 0 and two of the others 
are constants, or det(l. m.(A)) = 0. 

Let det(l. m.(A)) = 0. Denote A! = D • E\. . .Ek- 1, and consider two 
possibilities in accordance with our induction hypothesis. 

(i) 1. m.(A') has three constant entries. Let Ek = ( \ ?). There are three 
possibilities (up to a “symmetry”) for the matrix 1. m.(A'): 

(a) All the entries of 1. m.(A') are constants. In this case, the matrix 

1. m.(A) is of the form Q. The result follows. 

(b) Precisely three entries of 1. m.(A') are constants. In this case, one of 
them is 0. Let 1. m.(A') = ({ J). Then 1. m.(A) has three constant 
entries. 

(c) The same as (b), but 1. m.(A') = (J{), / nonconstant. Then 

1. m.(A) = ^ so that the first row of 1. m.(A) is 

1. m.(/) times the second row and the first column is 1. m.(g) times 
the second column. 

Now we come to the main case. 
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(ii) Let A f = ( ® J ) an d, sa Y> 

(1. m.(a), 1. m .(b)) = h • (1. m.(c), 1. m .(d)); (9.6) 

(1. m.(a), 1. m.(c)) = h' • (1. m.(&), 1. m.(d)) (9.7) 

for some nonconstant monomials /i, h' (if some of them are constant, then 
we are done). This means, in particular, that a, 6, c are nonconstant. 
Again, we consider only one case here, where Ek = ($1), g ^ 0. Then 

A = Since det(A) = 1, this yields 

1. m.(c) • 1. m .(b + ga) = 1. m.(a) • 1. m .(d + gc) = h • 1. m.(c) • 1. m.(d + gc ). 

Therefore, 1. m.(b + ga) = h • 1. m.(d -f <?c). This together with (9.6) means 
the first row of 1. m.(A) is h times the second row. Also, by (9.7), we have 
1. m .(b + ga) = 1. m.(ga) = 1. m.(g) • 1. m.(a), and 1. m .(d + gc) = 1. m .(gc) = 
1. m .(g) • l.m.(c); hence the second column of 1. m.(A) is 1. m.(g) times the 
first column. 

This completes the proof of Proposition 9.1.10. 

To complete the proof of Theorem 9.1.4, we recall from the proof of Theo- 
rem 9.1.1 that polynomial p is coordinate if and only if the row (di(p), d 2 (p)) 
can be completed to a matrix from GE2(P2)- Combining this with Propo- 
sition 9.1.10 yields the result. ■ 



9.1.2 Algorithm for detecting coordinate polynomials 

Given a polynomial p = p(x 1,0:2), we want to figure out whether or not it 
is part of a basis of the polynomial algebra P 2 . 

Step 1. Take the derivatives d\(p), 0^2 (p); denote qi = di(p), q t 2 — d 2 (p). 

Step 2. If the leading monomial (1. m.) of q\ is not divisible by the leading 
monomial of q 2 (or vice versa), then p is not part of a basis — this follows 
from Theorem 9.1.1 and Proposition 9.1.10. If 1. m.(#i) = h • 1. m.(q 2 ) or 
1. m.(q 2 ) = h • 1. m.(^i) for some monomial h, then we go on to 

Step 3. Set q[ = qi - h • q 2 or qf 2 = <?2 - h • <71, respectively. If, say, 
1. m.(gi) = 1, then p is part of a basis by Theorem 9.1.1. If 1. m.(^) = 0, 
then p is part of a basis if and only if 1. m.(^) = 1 (again by Theorem 9.1.1). 
If 1. m.(q[) ^ 0 or 1, then repeat Step 2 upon replacing qi with q[ or q 2 
with q' 2 , respectively. 

Since the maximum of the degrees of qi and q 2 decreases every time we 
apply Step 3, we can apply it at most d+ (d— 1) + * * * + 1 = d(d + l)/2 
times, where d is the degree of the polynomial p. Therefore, our algorithm 
has quadratic growth with respect to the degree of a polynomial. 

If our algorithm has revealed p to be a coordinate polynomial, then it 
is also easy to find a polynomial q such that p and q generate the whole 
algebra P 2 . All we have to do is to keep track of the transformations in 
Step 3 in the matrix form as follows: (<?i, q 2 ) (qu q 2 )E(h), where E(h) = 
{-hi) or ( l ~i ) • In the end, if the polynomial p is coordinate, we arrive 
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at (di(p), d 2 (p)) = (1, 0 ) - M, where M is a product of all the matrices E(h) 
that we have encountered. It follows that M is of the form ( d2 j ^ ) . 
Then, we conclude that si = di(q), s 2 = d 2 (q) for a polynomial q G P 2 , 
and p and q generate P 2 . 

Remark 9.1.11. This algorithm can also be used to find a sequence 
of elementary automorphisms that reduces p G P 2 to X\ in case p is 
a coordinate polynomial. Indeed, as we have just shown, we can find a ma- 
trix M G GE 2 (P 2 ) such that (di(p),d 2 (p)) = (1,0) • M together with 
a decomposition of M into a product of elementary matrices. Then, ap- 
plying the inductive argument from the proof of [404, Theorem 6] gives 
an effective procedure for constructing a desired sequence of elementary 
automorphisms . 

Namely, suppose M = £12 (pi) • £ 21 ( 91 ) £12 (p*) • E 2 i(q k ), where 

£12 (p) = ( 01 ) and E 2 i(q) = (J ?)• Then, from Wright’s proof, it follows 
that qk = qk(x 1 ) so that the matrix £ 2 i(#fc) is the Jacobian matrix of 
an elementary automorphism. Similarly, if the first elementary matrix on 
the right is of the form £i 2 (pfc), then pu = Pk{x 2 ), whence E\ 2 {pk) is the 
Jacobian matrix of an elementary automorphism. The inductive procedure 
now is clear. 

Remark 9.1.12. An algorithm alternative to that of Remark 9.1.11 can 
be obtained as follows. Given a coordinate polynomial p G P 2 , we can find 
a polynomial q such that p and q generate P 2 — see above. Then we can 
use the procedure based on [80, Theorem 8.5] to reduce (p, q) to (xi,x 2 ) 
by a sequence of elementary automorphisms. The difference between this 
algorithm and that of Remark 9.1.11 is (informally speaking) the same as 
between Nielsen’s and Whitehead’s algorithms in a free group (see Sections 
1.1 and 1.2). 

9.1.3 Relation to the Jacobian Conjecture 

In this section, we discuss very briefly how our method may be used to 
construct a possible counterexample to the Jacobian conjecture in higher 
dimensions. 

Proposition 9.1.13. If polynomials pi, . . . ,p n provide a counterexample 
to the Jacobian conjecture with minimal possible n, then each of those 
polynomials has outer rank bigger than 1. 

Proof. Suppose, say, pi has outer rank 1. Then, upon applying an auto- 
morphism (if necessary), we can assume that pi has the form pi = pi(aq). 
In order to have the Jacobian matrix invertible, we should have then 
Pi = a • x\ for some a G K*. Then, (n — 1) x (n — 1) Jacobian matrix J' = 
(i dj(pi)) 2 <ij< n is also invertible, which implies, by the minimality assump- 
tion, that K(xi)\p 2 , . . . 5 Pn\ = K(xi)[x 2 , . . . ,x n ], where K(x 1 ) is the quo- 
tient field of K[x 1 ]. It follows that £(xi,p 2 , . . . ,p n ) = K(x i,x 2 , . . . ,x n ), 
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which by Keller’s theorem [173] implies K[x\,p 2 , . . . ,p n ] = P n . Therefore, 
polynomials pi , . . . , p n generate the whole algebra P n and hence a contra- 
diction. ■ 

This shows that polynomials with unimodular gradient, but of outer rank 
bigger than 1, are the key to constructing a counterexample to the Jacobian 
conjecture. By a result of Suslin [379], every unimodular polynomial row of 
dimension n > 3 can be completed to an invertible nxn matrix over P n . Of 
course, the problem is to have those other (n— 1) rows satisfy the conditions 
dj(qi) = di(qj) for every row (<?i, . . . , q n ) — this is needed to make sure our 
invertible matrix is actually a Jacobian matrix. 

The higher dimension of our unimodular row, the “more room” we 
have for building a matrix with the desired properties. There are several 
algorithms known for completing a unimodular polynomial row to an in- 
vertible square matrix over P n (see, e.g., [224]), but all of them are rather 
complicated. A practical algorithm like that would be a major step to- 
ward constructing a counterexample to the Jacobian conjecture (if such 
a counterexample exists). 



9.2 Coordinates 

in Free Associative Algebra of Rank Two 

Here we use some results from the previous section to obtain informa- 
tion about automorphisms and coordinates of a free associative algebra of 
rank 2. We note that coordinates of a free associative algebra are usually 
called primitive elements (the same terminology is used in the theory of 
free groups). 

Let P 2 = K\x 1 , 0 : 2 ] be the polynomial algebra of rank 2 over a field K 
and A 2 = K(x 1 , 2 : 2 ) the free associative algebra of rank 2 over the same 
ground field. 

It is well-known that the automorphism groups Aut(P 2 ) and Aut(^ 2 ) 
are isomorphic. This is due to Makar-Limanov [233] (for K = C) 
and Czerniakiewicz [86] (for an arbitrary ground field). See also [80, 
Theorem 9.3]. 

Furthermore, there is a description of the group Aut(P 2 ) as a free product 
with amalgamation due to Shafarevich [334]; see also [80, Theorem 8.6], 
[94] , [404] , and references therein. 

All these results, however, reduce a structure of the group Aut(P 2 ) to 
that of smaller groups of automorphisms. In this section, we show that 
a simple argument leads to a more specific group-theoretic description of 
Aut(P 2 ) (as well as of Aut(A 2 )). We only use the additive group of the 
ground field K as a “building block” and then apply various group-theoretic 
constructions. 
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We call an automorphism (p G Aut(P2) an IL- automorphism if it is identi- 
cal on linear part (i.e., if it takes Xi to Xi+pi, where polynomials Pifi — 1,2, 
don’t have monomials of degree less than 2). A similar definition applies to 
automorphisms of A2. Denote the groups of IL-automorphisms of P2 and 
A 2 by Aut/ £, (-P2) and Aut respectively. The subgroup Autj L (P2) 
(or Autj L (A2)) is generated by elementary IL-automorphisms of the form 
{xi -* xi + /(x 2 ); x 2 x 2 } and {x\ -* x 2 -> x 2 + f(x 1)}, where 
one- variable polynomials / don’t have monomials of degree less than 2. 

Furthermore, let Aut°(P2) (respectively, Aut°(A2)) denote the group of 
augmentation-preserving automorphisms of P2 (or A2); these are automor- 
phisms of the form Xi — > Xi + pu where polynomials Pi, i = 1, 2, have zero 
constant terms. Then we have: 

Theorem 9.2.1. Let K be an arbitrary ground field. The group Aut°(A 2 ) 
is a semidirect product o/Autj^A^) and GL2(K) ( the subgroup Autjx(A2) 
being normal in Aut°(A2) and GL2 (K) a retract). The group Aut/^^) 
is the normal closure (in the group Aut°(A2)) of Aut e IL (A2). This latter 
group is isomorphic to the free product (K+)°° * (K+)°°, where (K+)°° is 
the direct sum of countably many copies of the additive group K + of the 
field K. 

All these statements remain valid upon replacing A2 with P2 . 

Thus, group-theoretic properties of the groups Autj L (P2) and Autf^^) 
(and these are the main building blocks of Aut(P2) and Aut(A2), re- 
spectively) are determined by properties of the additive group of the 
ground field K. This latter group may have a rather complicated struc- 
ture, but in some cases (especially when K is finite), its structure is fairly 
well-understood . 

Our further goal is to distinguish primitive elements of the algebra A 2 
(an element u G A 2 is called primitive if it is an automorphic image of x\ 
or, in other words, if there is a generating set {u,v} of A2). 

Based on the aforementioned isomorphism between Aut(P2) and 
Aut(^2), and also on results from the previous section on detecting 
generators of P2, we can prove: 

Theorem 9.2.2. There is an algorithm that distinguishes primitive 
elements of the algebra A2 over an arbitrary field of characteristic 0. 

Note that there is a very simple “commutator test” for deciding if a given 
pair of elements generates the algebra A2 — see [93] . The problem of dis- 
tinguishing primitive elements is obviously more difficult, yet our algorithm 
itself is fairly simple. 

Furthermore, driven by the desire to reveal nonprimitivity of an element 
of A2 just by inspection, we present a couple of very transparent necessary 
conditions for an element of A2 to be primitive. 

Denote by J2 the free special Jordan algebra of rank 2. This is a (nonasso- 
ciative) AT- algebra spanned by x\ and X2 with respect to a binary operation 
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x o y — \{xy + yx). To avoid a restriction char If ^ 2 , one can consider 
a somewhat less user-friendly definition of J2 upon replacing the binary 
operation given above by two operations: x — > x 2 and (x, y) — » xyx. 

Then we have: 

Proposition 9.2.3. For an arbitrary ground field K: 

(i) The algebra J2 is invariant under any automorphism of A2. 

(ii) The group Aut(J2) is isomorphic to the group Aut(A 2 ) {and, 
consequently , to Aut(P2))* 

Corollary 9.2.4. If u E A2 is a primitive element of A2, then u E J 2 . 

This corollary gives a very convenient criterion for an element of A2 to 
be primitive. Indeed, elements of J2 are characterized among the elements 
of A2 as follows (see [ 73 ] or [ 157 ]). Define an antiautomorphism ^ of A2 
that rewrites every monomial backwards. For example, (x^)^ = X2X1, 
{x\x2X\X2)*~ = X2X1X2X1, etc. Then is extended to the whole A2 by 
linearity. The elements u E A2 for which u^~ = u are called palindromic. 
Then we have [ 73 ] that an element u € A2 belongs to J2 if and only if it is 
palindromic. 

Thus our Corollary 9 . 2.4 gives a very convenient necessary (but not 
sufficient) condition of primitivity: 

Corollary 9.2.5. Primitive elements of A2 are palindromic {which means , 
incidentally , that every homogeneous component of a primitive element is 
palindromic ) . 

This condition is quite sensitive since the algebra J2 is very small 
compared to the enveloping algebra A2. 

We give here one more necessary condition of primitivity in A2 based on 
a result of [ 350 ]. Denote by A the augmentation ideal of A 2 (i.e., the set 
of elements without constant terms). Every element u E A has a unique 
presentation of the form u — di(u) • x\ + d2(u) • X2 (see, e.g., [ 80 ]). The 
elements di{u) are called (partial) Fox derivatives of u. Then we have: 

Proposition 9.2.6. If u E A is a primitive element of A2, then: 

(di(u))*~ ■ d 2 (u) = (d 2 (u))*~ ■ di(u). 

In other words, the element (di(u))*~ ■ d 2 (u) is palindromic. 

This condition is also not sufficient for primitivity, but it seems plausible 
that, together with the condition of Corollary 9 . 2 . 4 , it gives a sufficient 
condition for an element of the form x^+ {terms of higher degree) to be 
primitive. 
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9.2.1 Fox derivatives in a free associative algebra 

For an automorphism <p G Aut(P 2 ) that takes Xi to pi, i = 1, 2, the Jacobian 
matrix is defined as follows: J ^ = (dj(pt))i<i,j<2, where dj is the “usual” 
Leibniz partial derivation. 

Similarly, if ip G Aut(A 2 ) takes Xi to Ui , i = 1,2, then J \p = 
(dj(ui))i<ij< 2 , but this time dj denotes partial Fox derivation. (We use 
the same notation for Leibniz and Fox derivations without ambiguity.) 

There is a useful product rule for the Jacobian matrices (it is the same 
in commutative and noncommutative situations): 



* Jif)' (9.8) 

We are going to need some more background on Fox derivatives (the 
general reference here is [80]). 

The augmentation ideal A of the algebra A 2 is a free left and right A 2 - 
module with a free basis {xj, 1 < i < 2, so that for any u G A there is 
a unique presentation of the form u = d\{u) • x\ + d 2 (u) • x 2 as well as of 
the form u — x i • D\(u) + x 2 • D 2 (u). The elements Dj(u) are called right 
Fox derivatives of u, and dj (u) are left Fox derivatives. 

One can extend these derivations linearly to the whole A 2 by setting 

A(l) = di( 1) = 0. 

Then the result of [350] yields the following lemma: 

Lemma 9.2.7. Let u be a primitive element of A 2 . Then Di(u) • d 2 (u) — 
D 2 (u ) • di(u) = 0. 

Proof. It has been proved in [350] that for an automorphism (p G Aut(A 2 ) 
that takes Xi to yi, i = 1 , 2 , one has 

( D 2 (y 2 ) -D 2 {yi)\ (d x (yi) d 2 {y x )\ _ 

\-Di(y 2 ) D x {y x ) J \di(y 2 ) d 2 (y 2 )J ’ 

where c € K* and I is the identity matrix. 

It follows (see, e.g., [80]) that 

( d ^Vi) d 2{yi)\ ( D 2 {y 2 ) -D 2 (y x )\ _ 

U(j/ 2 ) d 2{y2))\-D x {y 2 ) D x { yi ) ) ~ c l ' 

whence d x (yi) ■ (~D 2 (yi)) + d 2 (y x ) • D x (y x ) = 0. This proves the claim. 



We also need Nagao’s theorem [287] (see also [404]): 

Theorem 9.2.8 ([287]). GL2 (-^[^]) = GIj 2 (A') *ut 2 (A') l'T 2 (A'[^]). where 
K[t] is the polynomial algebra in one variable t over K, and UT 2 is 
a group of 2x2 upper triangular matrices. The statement is also valid 
upon replacing the upper triangular group with the lower triangular group. 
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9.2.2 Proofs and algorithms 

Proof of Theorem 9.2.1. We start with the first statement. It is 
straightforward to verify that Aut/£ / (^2) is a normal subgroup of Aut°(A2). 
Also, it is obvious that the intersection of Aut/L(^2) with the group 
of linear automorphisms is trivial. This means the group Aut°(A2) is 
a semidirect product of Autj£,(A2) and GL <2 (AT). (The fact that Aut°(A 2 ) 
is a product of those two subgroups follows from the results of [86].) 

It follows that every automorphism ip G Aut°(A2) can be written in the 
form ip = Ao^, where A is a linear automorphism, ^ £ Aut/L(^2), and 
A o xjj means A(^). 

On the other hand, by the results of [86], every automorphism ip G 
Aut°(A2) can be written as a product of linear automorphisms and ele- 
mentary automorphisms of the form {x\ — )> X\ + c • £2 — ^ £2} an d 

{#1 — > X2 — y X2 H~ c • x™} for all possible c G K and m > 2. These 

elementary automorphisms clearly belong to the group Autj L (A2). Now 
a usual rewriting process (based on the equality abc = 6(6 _1 a6)c = ba h c) 
in combination with what was said in the previous paragraph proves the 
second statement of Theorem 9.2.1. 

Now we are going to prove the claim about the group G = Autj L (A2). 

Denote two copies of by A and B and their natural components 

by {A2, A3 , . . . } and {£2, £3, • . . }, respectively (we deliberately start with 
index 2 for subsequent notational convenience). All the groups A k and Bk 
are isomorphic to the group K + . 

We are now going to define a mapping r from A * £ into G on these 
components; the fact that a mapping like that can be extended to a homo- 
morphism of groups follows from categoric properties of the group-theoretic 
constructions involved. 

Let r take a* G Ai to the following automorphism a* : x\ x\ + a* • x\\ 
X 2 — > X 2 . Then, let r take bi G £* to the automorphism /%: x\ — > x\\ 
X 2 — > X 2 + h • x\. Everywhere, i> 2. 

First we prove that r is injective. By way of contradiction, suppose, say, 

&i • /3i a,k * fik = id, where &i (or $i) is a product of finitely many 

Oii. (or fa.), and id is the identical mapping. We assume that all d*, fa are 
nonidentical mappings. 

Then applying the product rule (9.8) for the Jacobian matrices yields 

1 ( d ot 1 ) • 'Pk(Jak) * Jp k = I } ( 9 - 9 ) 

where xfjj G Autj L (A2) are appropriate automorphisms (of no particular 
importance to us), and I is the identity matrix. 

All matrices xjjj ( . ) are obviously upper triangular, and matrices ( J§. ) 
are lower triangular. Furthermore, none of them is the identity matrix, and, 
moreover, none of them belongs to the group XJT2{K) since none of the J , 

J h do - 
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Consider now the abelianization x\ — » £; #2 — > t. An abelianized matrix 
or 'ip j(Jp.) a doesn’t belong to UT 2 {K) unless it is the identity 

matrix. 

Indeed, any off-diagonal entry in a matrix ip j {Jon), say, has the form 
ipj(f( x 2 ))? where / is some (nonconstant) one- variable polynomial, so that 
ipj(f(x 2 ))° — f{ipj{ x 2) a ) £ K since r ipj{x 2 ) must be either nonconstant or 
zero, so its abelianization is either nonconstant or zero, too. 

Applying Nagao’s theorem (see Theorem 9.2.8) to the abelianized equal- 
ity (9.9) yields a contradiction (note that J^ k ^ I in (9.9)) that completes 
the proof of r being injective. 

The fact that r is surjective follows from the very definition of the group 
Autj L (A 2 ). Thus, r is an isomorphism. 

The corresponding statements about automorphisms of P 2 can now be 
easily deduced from the fact that the groups Aut 0 (P 2 ) and Aut°(A 2 ) are 
naturally isomorphic. We omit the details. ■ 

Proof of Theorem 9.2.2. Let u E A2. Denote by u a E P2 the abelian- 
ization of u. If u a is not a coordinate polynomial of P 2 , then u is obviously 
not a primitive element of A2. Note that we can decide whether or not u a 
is coordinate using the algorithm from Section 9.1.2. 

Let u a be a coordinate polynomial of P 2 • Using again a procedure from 
Section 9.1.2, we can find a sequence of elementary automorphisms that 
takes x\ to u a . Apply the same sequence to x\ but in the algebra A2 (we 
identify elementary automorphisms of P 2 and A2 by means of the natural 
isomorphism mentioned in Section 9.2). If we arrive at the element u , then 
u is obviously primitive. What is not so obvious is what happens if we 
arrive at a different element; call it v. 

We are going to show now that if v ^ it, then u is not primitive in A 2 . By 
means of contradiction, suppose u is primitive. Let ip E Aut(^) take x\ 
to u. Furthermore, let <p(xi) = v in A2, so that <p{x\) = u a in P 2 (we use 
the same letter for an automorphism p E Aut(A 2 ) and its natural image 
in Aut(P 2 )). 

Since u a = v a , this yields p{x\) = xp(xi) in P 2 . By [80, Theorem 8.5], 
this implies ip = ip {a) for some a E Aut(P 2 ) of the form {x\ — > x\\ X2 -» 
x 2 + f{ x i)}* This means p = ip {a) also in Aut(A 2 ). 

But a, as well as its fellow automorphism of A2, doesn’t change x\\ 
hence ip{a){x 1 ) = ^(^l) both in P 2 and A 2 . Therefore, we have in A 2 that 
v = <p{x\) = ip{a)(x 1 ) = ip(x 1 ) = -u, a contradiction. 

Therefore, u was not primitive in A2, and this completes the proof of 
Theorem 9.2.2. ■ 

Proof of Proposition 9.2.3. 

(i) It is clearly sufficient to prove this statement for linear automorphisms 
and for automorphisms of the form {x\ — )> x\ + x\\ X 2 -» X 2 } and {x\ — > 
x\\ x 2 -> x 2 }, k> 2 . 
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For linear automorphisms, the statement is obvious since J 2 is a if-linear 
subspace of A 2 . 

Since any power of X\ and x 2 belongs to J 2 , we have (p(xi),(p(x 2 ) £ J 2 for 
any elementary automorphism <p. Now, by an obvious inductive argument, 
we have ( <p(uv + vu))*~ = <p(uv + vu), provided {<p{ u ))^ = <p(u) and 
= <p(v). Therefore, cp(uv + vu) belongs to J 2 by [73]. The case 
when char K = 2 is treated similarly. 

(ii) As we have just seen, every automorphism of A 2 induces an auto- 
morphism of J 2 ; this mapping is obviously injective. Conversely, if p is an 
automorphism of J 2 , then <p(xi) and p(x 2 ) generate the algebra J 2 and 
hence also generate A 2 . The result follows. ■ 

Proof of Proposition 9.2.6. We are going to show that there is the fol- 
lowing “mirror symmetry” between left and right Fox derivatives for any 
u £ A 2 (it actually holds in a free associative algebra of arbitrary rank): 

Di(u*-) = ( di{u)Y ", (9.10) 

Without loss of generality, we can assume u e A, so let u = J2di(u) • Xi. 
Then u = Yl x i ’ an d hence (di(u))*~ = Di(u*~), proving the 

equality (9.10). 

Combining (9.10) with Lemma 9.2.7 and Corollary 9.2.5 yields the result. 



9.3 Density of Coordinates in Polynomial Algebras 

The exposition in this section follows [107]. 

Let K be a field of characteristic 0. In this section, we establish the 
following amazing fact: the set of coordinate polynomials of K[X] = 
K[x 1 , . . . ,x n ] is “dense” in the set of polynomials of the form (x* + #), 
d eg(q) > 2: 

Theorem 9.3.1. For any polynomial h oftheform(xi+q), deg (q) > 2, and 
any integer m > 2, there is a coordinate polynomial p such that deg (p—h) > 

m. 

This contrasts sharply with the situation in other, noncommutative al- 
gebraic systems; for example, the set of primitive elements (that is what 
generators are usually called in a noncommutative setting) of a free Lie al- 
gebra is not dense (see, e.g., [105]). Moreover, although the automorphism 
groups of K[x i,x 2 ] and K(x i,x 2 ) (the free associative algebra of rank 2) 
are isomorphic, we have: 

Proposition 9.3.2. The element u = x 1 + x\x 2 cannot be completed to 
a primitive element of K(x i,x 2 ) by monomials of degree higher than 2. 
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Our proof of Theorem 9.3.1 is based on some results of Anick [16] 
and that of Proposition 9.3.2 on a characterization of automorphisms of 
K(x i,#2) as it appears in [350]. 

First we have to give some more background on polynomial automor- 
phisms. Anick [16] proved that, with respect to the formal power series 
topology (Zarisky topology), the set J of endomorphisms of K[X ] with 
an invertible Jacobian matrix is a closed set, and the group of tame au- 
tomorphisms of K[X ] is dense in J. This means that for any polynomial 
mapping F = (/i, . . . , f n ) with 0 ^ J(F) G K and for any positive integer k 
there is a tame automorphism G = (gi,-.-,g n ) such that the polynomi- 
als ( fi — gi) contain no monomials of degree less than k. Bryant, Drensky, 
and C. K. Gupta [61, 105] developed a technique based on representation 
theory of the general linear group GL n (K) to study the automorphisms of 
some (commutative and noncommutative) algebras. An interpretation of 
the result of Anick in this language is given in [102, Theorem 4.2.7]. We 
recall some details here briefly. 

Let Pk be a AT-vector space of all homogeneous polynomials of degree 
k > 0. Let ifc, k > 2, be the semigroup of all polynomial endomorphisms 
F = (/i> • • • j /n) such that Xi is the only monomial of fi of degree less 
than k] i = 1, . . . , n. We write 



fi 



+ 9i + hi , 



where gi G Pk is the homogeneous component of fi of degree fc, and hi has 
no monomials of degree < k. The group GL n (AT) acts on Ik by conjugation. 

It turns out that there is a homomorphism p of Ik onto the direct sum of 
additive groups P® n = Pk 0 • • • 0 Pk such that ip(F ) = (gi, . . . , g n ). It has 
been established that the action of GL n (AT) induced by (p on p{Ik) gives 
a structure of a GL n (AT)-module to (p(Ik)- 

Let T be the group of tame automorphisms of the algebra K[X], and let 
S k be the set of all polynomial mappings S = (si, . . . , s n ) G Ik such that 
Si = Xi + gi + hi, where gi G Pk and hi G Yli>k Fu with the property 



E 

2=1 



% 

dxi 



= 0 . 



The main step of Anick’s proof (see [102], Step 2 of the proof of 
Theorem 4.2.7) is to show that 

ip(Tni k ) = (p( Sfc). 

This yields the following lemma: 

Lemma 9.3.3. For any S G S& C Ik, there is a tame automorphism Gk 
of K[X] such that S o G ^ 1 is in Ik+i- 

Proof of Theorem 9.3.1. Let u be any polynomial without constant and 
linear terms, and let Ui be the homogeneous component of degree i of the 
polynomial u; i = 2 , . . . , m. Let k be the smallest integer such that Uk 7^ 0. 
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Write the partial derivative of u k with respect to x i in the form 



o fc— 1 
dxi ^ 

J=0 



a,jX 



3 

25 



where polynomials dj do not depend on x 2 . There is a homogeneous 
polynomial w k G P k such that 



dwk 

dx 2 



k - i 



Yl a i x 2 - 

J=0 



Consider an endomorphism F& of the algebra K[X] defined by 






Clearly, Fk G Ik and, because of the choice of w k , also Fk G Sfc. Hence, 
by Lemma 9.3.3, there is a tame automorphism Gk = (gi,&, . . . , <7 n ,fc) G Ik 
such that Ffc+i = F k o G^ 1 G Jfc+i- Therefore, gi, k = x\ + u k + u^+i + v, 
where is homogeneous, of degree k + 1, and v G p i- 

Continuing this way, we obtain a tame automorphism G k + i G J/c+iF 
such that gifi+i = + (wfc+i - Vfc+i) + w, where w G Xw>fc +2 p i- 

If we act by the automorphism on the polynomial we get a poly- 
nomial of the form u k + s, where s has no homogeneous components of 
degree less than (k — l) + (fc + l) = 2 k > fc+1. Therefore, the automorphism 
G k +i o Gfc takes x\ to x\ + u k + tifc+i + (terms of higher degree). 

In a finite number of steps, we obtain a tame automorphism G = 
(#!,..., # n ) such that 

9i = Xi + u -f (terms of higher degree). 



This completes the proof of Theorem 9.3.1. 



Proof of Proposition 9.3.2. By way of contradiction, suppose there is 
an element w G A 3 such that u = x\ + x\x 2 + w is a primitive element of 
K(x i,x 2 ). 

By Lemma 9.2.7, we have 

d 2 (u) • Di(u) - di(u) • D 2 (u) = 0. (9-11) 

Now evaluate: di(u) = 1 + d\(w) = 1 + x\ (mod A 2 ); d 2 (u) = x\ + 
d 2 (w) = xi (mod A 2 ); Di(u) = 1 + x 2 + Di(w) = 1 + x 2 (mod A 2 ); 
D 2 (u) = D 2 (w) = 0 (mod A 2 ). 

Plugging this into (9.11) gives 

x \ (1 + x 2 ) = 0 (mod A 2 ), 



a contradiction. 



A combination of Theorem 9.3.1 and Proposition 9.3.2 calls for an ex- 
ample of a coordinate polynomial p G K[x i,x 2 \ of the form x\ + x\x 2 + 
(terms of higher degree); an example like that is given below. 
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Example 9.3.4. The following polynomial;? = x\+x\X 2 -\-\(x\x\ — x\x<i — 
x\ + x\) — jq{%i + X2) 4 is coordinate since it is the image of x\ under an 
automorphism af3a~ 2 f 3a, where a takes x\ to (pc\ + X 2 ) and fixes # 2 , and 

/ 3 fixes x\ and takes x<i to (X 2 — ^). 



9.4 Coordinates in K [z] [x, y] 

The exposition in this section follows [112]. 

A well-known theorem of Jung and van der Kulk [164, 194] says that 
all automorphisms of the polynomial algebra K[x,y] in two variables over 
a field K are tame; i.e., they can be decomposed as products of affine and 
triangular automorphisms. A combinatorial description of A\it(K[x,y]) as 
the amalgamated free product of the subgroups of affine and upper triangu- 
lar automorphisms is also well-known (see the book by Cohn [80]). Several 
algorithms have been discovered that determine whether a homomorphism 
of K[x,y] is an automorphism and, if it is, decompose it as a product of 
linear and triangular automorphisms (see [69, 80, 119]). Results of [359] 
over a field of characteristic 0 give that one can also determine whether 
a polynomial p(x,y) G K[x,y] is a coordinate (i.e., if it is the image of x 
under some automorphism P = (p,q) of K[x,y]) and to find a concrete P 
with this property. 

On the other hand, very little is known about automorphisms of the poly- 
nomial algebra R[x,y], where R is a commutative algebra. The first result 
in this direction was the famous example of Nagata [288] of an automor- 
phism of the K [^-algebra AT[z][x, y], which is not tame. Then Wright [404] 
described the structure of the group of tame automorphisms of R[x, y] over 
any principal ideal domain as an amalgamated free product in the same 
way as over a field. Wright showed also that the group of all automorphisms 
of i?[x, y] is an amalgamated free product of the affine group and one more 
group of automorphisms that, when R is not a field, properly contains the 
group of upper triangular automorphisms. 

Recently, Lam and Yu [200] have obtained algorithms: (i) for recognizing 
coordinates of D[x,y], where D is a unique factorization domain; (ii) for 
recognizing tame coordinates of R[x, y], where R is an Euclidean domain. 
These algorithms are applicable, in particular, to polynomials from Z[x, y\. 

The purpose of this section is to study K [z ] -automorphisms and coordi- 
nates of K [z][x, y], where K is a field of characteristic 0. Applying results 
of [404] for jRT[z][j;, y] and of [359] for K[x,y], we characterize tame coordi- 
nates of K[z][x,y]. Specifically, a polynomial p(x,y) G AT[z][x,y] is a tame 
coordinate if and only if 1 can be obtained from the partial derivatives 
Px = dp/dx and p y = dp/dy using the Euclidean algorithm. Hence the 
problem of whether a polynomial p(x,y) G K[z][x,y\ is a tame coordinate 
can be solved by using division of polynomials only. In particular, our al- 




148 9. Coordinate Polynomials 



gorithm produces a concrete tame automorphism of K[z][x,y] sending x 
to p(x,y). We also use recent results of Daigle and Freudenburg [87] on 
locally nilpotent derivations and characterize coordinates of #[ 2 ] [a;, y] (i.e., 
images of x under automorphisms of K[z][x,y]). Namely, we show that 
p(x , y) E K[z] [x, y] is a coordinate in K[z] [x, y] if and only if p(x, y) is a co- 
ordinate as an element of K(z)[x, y] and has a unimodular gradient, which 
means that if [ 2 :] [a:, y] is generated as an ideal by the partial derivatives p x 
and p y . As in the tame case, our approach allows us to solve effectively the 
problem of whether a polynomial is a coordinate, but this time we have to 
apply Grobner basis techniques instead of the Euclidean algorithm. 

The Nagata automorphism was recently proved to be wild when con- 
sidered as an automorphism of the polynomial algebra K[x,y,z] in three 
variables; see [343]. On the other hand, M. Smith [370] discovered another 
important property of the Nagata automorphism. She showed that it is 
stably tame and becomes tame if we extend it to an automorphism of 
K[z][x,y,t] fixing t. 

Considered as an automorphism of K(z)[x, y], the Nagata automorphism 
is a conjugate of an elementary automorphism by another elementary 
automorphism. Hence, the Nagata automorphism is a product of three 
elementary automorphisms of K(z)[x,y\. We study automorphisms of 
K [z][x, y] that have a similar presentation. In particular, we consider poly- 
nomials of the form p(x,y) = x + q{a(x) + by ), where q(w) E if[z][u;], 
a(x) E if[z][x], b E K[z], and study the problem when these polynomials 
are coordinate in K[z][x,y\. We establish that if q(w) is divisible by all 
irreducible factors of 6(z), then p is coordinate and is the image of x under 
an automorphism that is a product of three elementary automorphisms of 
K(z)[x, y\. In the special case where q(w) is divisible by b(z) itself, we show 
that p is really a “Nagata- like” coordinate and is the image of an automor- 
phism of K [z][x, y] that is a conjugate of an elementary automorphism by 
another elementary automorphism of K(z)[x,y]. We also show that auto- 
morphisms with q(w) divisible by b(z) are stably tame. In this way, our 
results give a new large family of automorphisms of AT[z][x,?/] that may 
be considered as candidates of wild automorphisms of K[x,y,z]. In fact, 
Umirbaev and Yu [393] have recently proved the following 

Theorem 9.4.1 ([393]). Any wild coordinate of K[z][x,y] is also a wild 
coordinate of K[x,y, z\. 

This result therefore enhances the importance of producing wild 
coordinates in AT[z][x,y]. 

9.4-1 Preliminaries 

We fix a field K of characteristic 0 and, if not explicitly stated, consider 
commutative unitary AT-algebras only. If R is an algebra, we denote by 
R[x,y] the polynomial algebra in two variables x, y over R. Sometimes we 
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use the notation p x and p y , respectively, for the partial derivatives dp/dx 
and dp/dy of p E R[x,y], In what follows, we assume that R is one of 
the algebras K , K[z], or K(z). We denote the endomorphisms of R[x,y] 
as F = (fi(x,y), f 2 (x,y)) assuming that fi and f 2 are respectively the 
images of x and y (and that F is an i?-algebra endomorphism). If F is an 
automorphism, then its inverse is denoted by F~ l . We use similar notation 
for endomorphisms of polynomial algebras in more than two variables. The 
Jacobian matrix of F = (/i, f 2 ) E Endi?[x, y] is 



Jf = = 



'dfi/dx dfi/dy\ 
df 2 /dx df 2 /dy ) ’ 



The composition FoG of two endomorphisms F = and G = (< 71 , g 2 ) 

is 



FoG = F{G) = {h(g l ,9 2 ),f 2 (g 1 , 92 )). 

The chain rule gives that 

JFoGfay) = JF(9u92)JG{x,y). 

In particular, if F is an automorphism, then its Jacobian matrix is invertible 
over R. 

Now we refresh some definitions. 

Definition 9.4.2. An automorphism of R[x, y] is called tame if it belongs 
to the subgroup of Aut(i?[x, y]) generated by affine automorphisms and 
triangular automorphisms, the latter being defined as 

F = {ax + f{y),(3y), 

where a, (3 E R* and / E R[y] is a polynomial that does not depend on x. 
Automorphisms that are not tame are called wild . 

Remark 9.4.3. The notion of tame and wild automorphisms depends 
on the ground algebra R. For example, Nagata [288] constructed the 
automorphism of K[x, y , z] defined by 

N=(x- 2 (y 2 + zx)y - ( y 2 + zx) 2 z, y + (y 2 + zx)z, z), 

which is wild when considered as an automorphism of K[z][x,y]. Only 
recently was it proved [343] that N is wild as an automorphism of K[x, y, z]. 

Definition 9.4.4. Any automorphic image p = p(x,y) E R[x,y] of x is 
called a coordinate polynomial or simply a coordinate. If P = (p, q) for 
some tame automorphism P of i?[x, y], then p is a tame coordinate. If all 
automorphisms P that send x to p are wild, then the coordinate p is also 
called wild. 

Remark 9.4.5. If p is a tame coordinate of R[x,y], then all automor- 
phisms Q of R[x,y] with the property Q = (p,q) are tame. Indeed, if 
P — (p,qo) for some tame automorphism P of R[x,y] and Q = (p,q) E 
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Aut(j R[x, y ]), then the automorphism S = Q o P~ l fixes x. It is easy to see 
that 5 = (x,ay + g(x)) for some a G R* and g(x) G R[x]\ i.e., S is tame. 
Since P is also tame, this implies that Q is tame. 

Let GE 2 (R[x,y]) be the subgroup of GL 2 (R[x,y}) generated by elemen- 
tary and diagonal matrices. By the Jung-van der Kulk theorem and the 
chain rule, the Jacobian matrix of every automorphism of K[x,y] belongs 
to GE 2 (K[x, y]). On the other hand, the weak Jacobian theorem of Wright 
states that if the Jacobian matrix of an endomorphism F of K[x, y] belongs 
to GE 2 (K[x,y\), then F is an automorphism. This result was general- 
ized to coordinate polynomials by Shpilrain and Yu [359]. They showed 
that p(x, y) G K[x,y\ is a coordinate polynomial if and only if the vector 
(dp/dx, dp/dy) is the first row of some matrix in GE 2 (K[x, y]). An equiva- 
lent form of this statement is that ( dp/dx , dp/dy) can be brought to (1,0) 
by a Euclidean algorithm. 

The results of Wright [404] and Shpilrain and Yu [359] are based on 
the description of the group GE 2 {K[x\, . . . ,x n ]) as an amalgamated free 
product. Recall the definition of an amalgamated free product (see, e.g., 
[231] or [404]). If A, B, and C are three abstract groups such that <p: B A 
and 0: B C are two embeddings, then the amalgamated free product 
G = A*b C is generated by A U C, and the defining relations of G are the 
defining relations of A and C together with the defining relations tp(b) = 
-0(6), b G B. If we consider B as a subgroup of A and C and assume that 
An C = B, then the elements of G = A C can be presented as 

g = ba £ 1 1 cia 2 c 2 . ..a k c £ k 2 , 

where b G B, a* G A, c* G C, and e 1 , e 2 are equal to 1 or 0, depending on 
whether or not a\ and c k occur in the expression of g. It is well-known that 
g is different from 1, provided that ai, . . . , a k and ci, . . . , c k do not belong 
to B. The description of GE 2 (K[xi, . . . ,£ n ]) given by Wright [404] is the 
following. 

Theorem 9.4.6 ([404]). The group GE 2 {K[xi, . . . ,x n ]) is isomorphic to 
the amalgamated free product of the subgroup GL 2 (K) and the subgroup 
B 2 (K[xi, . . . , x n ]) of all lower triangular matrices with polynomial entries. 

We assume that we can perform concrete calculations with the elements 
of K. We also introduce an arbitrary ordering on the monomials of K[x,y,z\ 
that allows us to make induction and is preserved under multiplication. For 
example, we may consider the usual lexicographic ordering or the deg-lex 
ordering, comparing the monomials of K[x,y,z) first by total degree and 
then lexicographically with x > y > z. (See, e.g., [13] for different orderings 
of K[x 1 , . . . , x n ].) We say that a matrix 

a =(T “ 12 )eGE 2 (K[x,y,z}) 

\tt 2 i a 22 J 
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can be decomposed into a product of diagonal and elementary matrices 
using a Euclidean algorithm if, in each step of bringing a to its diagonal 
form by elementary transformations, we multiply it on the right by an 
elementary matrix that decreases the leading terms of the first row of a. 
For example, if the leading term of an is equal to the leading term of ban 
for some monomial b G K[x, y , z], then we are allowed to replace a by 

ua=( an ai2 )(\ ?W Cl1 Cl2 V 

\fl21 a 22/ \—b 1/ \C2i C 22 J 

In particular, if / and g are two relatively prime polynomials in K[x, y, z] 
and we reach 1 by applying the usual Euclidean algorithm to / and < 7 , then 
the corresponding operations can be written in a matrix form as 

(an 012 ) ei . . .e k = a (l 0) , a€K*, 

where ei , . . . , e*, are (alternatively lower and upper) triangular matrices 



e* 



1 hi 
0 1 



1 — 



1 

hi -\- 1 




We shall make use of a result of the thesis of Park [297]. Since the journal 
version of [297] has not been published yet, a proof for the case of polyno- 
mials in two variables can be found in [359] . A careful study of the proof 
in [359] shows that it works for any number of variables and any ordering. 

Proposition 9.4.7. Every matrix in GE 2 (K[x,y,z]) can be decomposed 
into a product of diagonal and elementary matrices by using a Euclidean 
algorithm only. 

Remark 9.4.8. If (/,#) is a pair of polynomials brought to (1,0) by the 
Euclidean algorithm in k steps 

(an a 12 )e 1 ...e k = a(l 0) , a G AT*, 

then the matrix e \ . . . e k belongs to GE 2 (K[x, y , 2 ]), and one presentation 
as an element of GL v(K)*b 2 (k) & 2 (K[x, y , z]) can be obtained by replacing 
the upper triangular matrices (en + 622 ) + ^612 by the products ((en + 
^ 22 ) + hie 2 i)(ei 2 + e 2 i) if hi is not a constant. 



9.4-2 Characterization of tame and wild coordinates 
in K[z][x,y] 

In this section, we characterize tame coordinates in K[z][x,y]. Then we 
give algorithms that recognize whether a polynomial p(x,y) G K[z][x,y\ 
is a tame coordinate and, if it is, find concrete tame automorphisms send- 
ing x to p(x,y). Finally, we characterize all coordinates in K[z][x,y] and 
give a procedure that determines whether a polynomial is a coordinate 
of K[z][x,y\. In particular, we are able to find effectively a lot of wild 
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automorphisms of if[z][x, y], giving in this way new candidates for wild au- 
tomorphisms of K[x, y, z\, all of them fixing z as in the example suggested 
by Nagata. 

To prove our main results on tame automorphisms, we modify the con- 
siderations in the proof of the weak Jacobian theorem in [404] and the 
description of the coordinate polynomials in K[x,y ] in [359]. 

Lemma 9.4.9. Let £ : K[x, y, z] — > Z 3 be the degree function on K[x , y, z] 
induced by the lexicographic ordering x > y > z; i.e., £(/) = (di,d2, ^3) if 
the leading term of f ^0 is ax dl y d2 z d3 , a G K* . Let ai, . . . , a& be matrices 
in GL2 (K) that do not belong to the lower triangular group B2(K), and let 
c% = {en + e 2 2) + /<e 2 1, i = 1, . . . , fc, where /1, . . . , /* are polynomials of 
positive degree. Then the row-matrices 

(UO Vo) = (l 0) , (Ui Vi) = (Ui - 1 Vi- 1 ) fliCi, 

i = 1, . . . , k, satisfy the equation 

Z(Ui) =t(vi) + £(/*)• 

Proof. We use induction on i. If a\ = anen + ai2ei2 + 0:21^21 + 0:22^22, 
ol\2 7^ 0, then direct calculations show that 

u\ = an + 012/1, vi = ai 2 , 

and £(ui) = £(/i), £(vi) = £(012) = (0,0,0). Similarly, if a* = /?ne n + 
012^12 + 021^21 + 022^22 •> 012 7^ 0, then 

Vi = (012 fi + /?ll)^i-l + (022 fi + V* = 012V>i-i + 022V i-l, 

and £(t/i) = £(w*-i) +£(/i), £(^i) = £(ui-i), which completes the proof by 
induction. ■ 

The following theorem is one of the main results of this section. 

Theorem 9.4.10. The following statements for p(x,y) G K[z][x,y] are 
equivalent: 

(i) The polynomial p(x,y) is a tame coordinate of K[z][x,y], 

(ii) There exists a matrix g G GE 2 (K[x,y,z]) such that 

(px Py) 9 = (1 0 ) ; 

i.e., (px p y ) can be brought to ( l o) by elementary transformations. 

(iii) When applying a Euclidean algorithm top x andp y , the result is equal 
to 1. 

Proof. The equivalence of (ii) and (iii) follows immediately from Proposi- 
tion 9.4.7. The implication (i) => (ii) is a consequence of the chain rule and 
the fact that K[z\ is a Euclidean domain (hence GE 2 (K[z]) = GL 2 (K[z]) 
and the Jacobian matrix of any affine automorphism of if[^][x,y] belongs 
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to GE2 (K[x,y,z])). Therefore, it is sufficient to show that (ii) implies (i). 
Let 



(Px Py ) 9 = (1 0 ) 

for some matrix g = g(x,y) G GE 2 (K[x,y,z]) (depending on x,y and on 
the “constant” z) and let 

g = aiCia 2 C2 . . . 

where ai, . . . , a* G GL 2 (K) and a 2l . . . , do not belong to the lower 
triangular group B 2 (K ), C{ = (en + 622) + f%e 21 for some nonconstant 
polynomials fi G K[x, y, 2], i = 1 , . . . , fc, and e = 0 , 1 . Since 

(» »)(/, ?)=(! °) 

implies that (uv) = (10), without loss of generality we may assume that 
e = 0 . Let 

a i = “ 12 ) . 9(x,y) = aigi(x,y), 

\ a 21 a 22 J 

q(x, y) = p(e*iix + ai 2 y, a 2X x + a 2 2 j/)- 

Replacing x and y, respectively, by anx + OL\ 2 y and c*2ix + a 22 y in 
(p* p y )g(x, y) = (10), easy computation shows that (p*Pv)ai goes to 
(qx q y ) (i.e., to the gradient of q(x, 3/)), and we obtain a new equation 

(q x 9y)g'{x,y) = (1 0), g'{x,y) = gi{a u x + a 12 y,a 2 ix + a 22 y), 

where the decomposition g f = c\a 2 . . . Ck-idk starts with c\. Let £ be the 
degree function on K[x,y,z] induced by the lexicographic ordering x > y > 
z. Since 

(& %) = (! °) H l °k-i ■ ■ ■ a 2 lc r 1 

andcjf 1 = (en+e 2 2)-/ie 2 i, Lemma 9 . 4.9 gives that £(q x ) = £(q y )+€(fi). 
First, we assume that f\ essentially depends on x. Let, for example, £(q y ) = 
(di,d 2 ,d 3 ), £(/i) = (ei,e 2 ,e 3 ), and e x > 0 . Then £(q x ) = (di + ei,d 2 + 
e 2 ,d s + e 3 ). Since q xy = q yx , we obtain that £(q xy ) = (di + ei-l,d 2 + 
e2,d 3 + e 3 ) and^(g yx ) = (di,d 2 -l,d 3 ) if d 2 > 0, or £(q yx ) < (di-l,d£,d£) 
for some d f 2 ,d f 3 if d 2 = 0 . Since this is impossible, we derive that fi does 
not depend on x. Let f(y) G iT[z][y] be a polynomial such that f y — f\. 
Replacing x by x + f(y) in the equation ( Qx q y ) g x = ( i o ), we obtain that 
(qx g y ) a goes to the gradient of pi(x, y) = q(x + f(y ), y). In this way we 
obtain the equation ( (pi)* (pi) v ) g 2 = ( 1 o ), where g 2 = a 2 c 2 . . . Cfc-ia*.. By 
inductive arguments, we may assume that pi(x,y) is a tame coordinate. 
Since both A\ = (anx + oti 2 y, a 2 ix + a 22 y) and F\ = (x + f(y),y) are 
tame automorphisms, we obtain that p(x, y) is also a tame coordinate. 
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Remark 9.4.11. The theorem of Wright about the structure of the group 
GE 2 (K[z][x, y ]) holds for any Euclidean domain R in place of K[z\. Never- 
theless, the proof of Theorem 9.4.10 does not hold for Euclidean domains R 
of characteristic 0 that do not contain the field Q. The reason is that in the 
inductive step of the proof we have used that the integral of the polynomial 
f(y) G if[z][y] also belongs to if[z][y]. For instance, Theorem 9.4.10 is no 
longer true even for R = Z. A simple example is the following. If we replace 
in the Nagata automorphism 

N = (x - 2 (y 2 + zx)y - (y 2 + zx) 2 z , y + (y 2 + zx)z) G Aut K[z\ [x, y] 
z by 2, we obtain the automorphism 

No = ( v , q) = (x- 2 {y 2 + 2 x)y - 2 (y 2 + 2a;) 2 , y + 2 (y 2 + 2a:)) € Aut Z[x, y], 

and p x = 1 - 4(y + 2 (y 2 + 2a:)), p y = -4 y(y + 2 (y 2 + 2a:)) - 2 (y 2 + 2x), 

Py ~ VPx = ~y~ 2 (y 2 + 2x), p x - 4 (p y - yp x ) = 1. 

Similarly, q x = 4, q y = 1 + 4 y, q y — yq x = 1. Hence both p and q satisfy the 
hypothesis of Theorem 9.4.10 (iii). On the other hand, we shall see that 
N 0 is a wild automorphism of Z[rc,y]. Following the proof of Wright [404, 
p. 247] for existence of wild automorphisms of i?[x, y] where R is a principal 
ideal domain that is not a field, our considerations are the following. Let 

I = {A = (a t x + fay, a'iX + /?•?/)} 

be a set of nontrivial left coset representatives of 

GL 2 (Q) = {F = (ax + (3y , olx + /3'y) | a, /?, a', (3' G K , Jp invertible} 

considered as a subgroup of Aut(Q[x, y]) modulo J5 2 (Q) = {(ax+/3y, a'x) | 
a, a' G Q*, /? G Q}. (Comparing with [404], we change left and right for 
cosets and lower and upper for triangular automorphisms because of the 
difference in the definitions of F o G and Jp-) As in [404], we can choose / 
as a subset of GL 2 (Z). Let also 

J = {C = (x + y 2 f{y),y) | / € Q} 

be the set of triangular automorphisms of Q[x, y] corresponding to polyno- 
mials without linear components. Then the set W of all automorphisms 

P = Ci o A\ o C 2 o • • • o Ak~ i oCfeoB, 

such that Ai e I, Ci € J, B € R 2 (Z), and C\ o A\ o • • • o i o Ci is not 
an automorphism of Z [x,y\ for / < fc, is a subgroup of Aut(Z[#,y]). The 
subgroup Aut°(Z [x,y]) of all automorphisms preserving the augmentation 
of Z[x, y\ has the presentation 

Aut°(Z[a:,y]) = GL 2 (Z) *b 2 (Z) W. 

In particular, if P = C\ o A\ o • • • o A^-i o Ck G W and k > 1, then P 
is a wild automorphism of Z[x, y\. In our case, we choose I in such a way 
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that it contains A = (x, y 4- 4x), we define C = (x + y 2 / 2, y), and a direct 
calculation shows that No = C~ 1 oAoC. Hence 7V 0 is a wild automorphism 
oiZ[x,y\. 

Corollary 9.4.12. There is an algorithm that determines whether a poly- 
nomial p(x,y) G K[z][x,y] is a tame coordinate and , if the answer is 
affirmative, produces a concrete tame automorphism of If [2] [a;, y] that 
sends x to p(x, y). 

Proof. Given a polynomial p G K[z][x,y\, we want to decide whether 
it is a tame coordinate. The algorithm can be derived from the proof of 
Theorem 9.4.10. We fix some ordering in K[x,y,z\ (e.g., the lexicographic 
or the deg-lex ordering satisfying x > y > z). 

Step 1. Take the partial derivatives q\ — p x , #2 = Py 

Step 2. If the leading monomial (1. m.) of q\ is not divisible by the leading 
monomial of <72 (or vice versa), then p is not a tame coordinate. If 1. m.(#i) = 
h • 1. m.(g 2 ) (or 1. m.^) — h • 1. m.(#i)) for some monomial h G K[z][x,y], 
then go to: 

Step 3. Set q[ = q\ — hq 2 and q 2 = q \ 2 (or q[ = q\ and q' 2 = #2 — hq\). 
If, say, 1. m.(^) = a G if*, then p is a tame coordinate of K[z][x,y\. If 
1. m.(<7i) = 0, then p is a tame coordinate if and only if 1. m .(^ 2 ) = ol G if*. 
If 1. m.((^) ^ if, then repeat Step 2 upon replacing q\ with q[ and q 2 with 

<?2- 

The algorithm for constructing a tame automorphism of K[z][x,y] that 
sends x to p is also contained in the proof of Theorem 9.4.10. For simplicity, 
we assume that we always know that p = p(x, y) is a tame coordinate. 

Step 1. Define an automorphism T = (ti(x,y),t 2 (x,y)) = (x,y) of 
K[z][x,y\. 

Step 2. Take Q = {qi,q 2 ) = (p x ,P y )- 

Step 3. If one of the polynomials q\ or <72 is equal to a nonzero constant a , 
then p = ax + f(y) (or p = ay + f(x)). Replace T by T = {t\{ax + 
f(y),y),h(ax + f(y))) (or by T = (hfaay + f(x)),t 2 (x,ay + f(x)))). 
The automorphism T is the inverse of an automorphism of if[z][:r,y] that 
sends x to the initially given p(x, y). 

If neither q\ nor (72 is a constant, then let the result of the division of 
qi by q 2 be qi = hq 2 + r (or q 2 = hq 1 + r). Then h = h(y) G K[z][y\ 
does not depend on x (or h — h(x) G if[z][x] does not depend on y). 
Choose f(y) G if[^][y] such that f y = h(y) (or f(x) G if[2:][x] such that 
f x = h(x)). Replace p(x,y) and T, respectively, by p(x + f{y),y) and 
T - (hiax + f(y),y),t 2 (ax + f(y),y )) (or respectively by p(x,y + f(x)) 
and T = (ti(x,ay + f(x)),t 2 (x,ay + f(x)))). Then go to Step 2. 

If we want to determine all automorphisms sending x to p(x, y), we can 
use Remark 9.4.5. ■ 

Now we apply our results to get similar algorithms in the free associative 
algebra over K[z)\ 
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Corollary 9.4.13. There is an algorithm that determines whether a poly- 
nomial p(x, y) in the free associative algebra K[z](x, y) is a tame coordinate 
and , if the answer is affirmative, produces a concrete tame automorphism 
sending x to p(x, y). 

Proof. By the proof of Theorem 1.2 in [360], for every field R and 
every coordinate qo{x,y) G R[x,y] there exists a unique coordinate 
qi(x,y) G R(x,y) that maps onto qo(x,y) under the natural homomor- 
phism R(x,y) — )• R[x,y]. Since every tame automorphism of 
is induced by a tame automorphism of K[z](x,y), this implies that if 
Po(x,y) G AT[z][x,p] is a tame coordinate, then it is an image of a unique 
tame coordinate pi(x,y) G K[z](x,y). Our algorithm for recognizing the 
tame coordinates in K[z](x,y) is the following. If p{x,y) G K[z](x,y), take 
its image po under the natural homomorphism K[z](x,y) -» K[z][x,y\. 
Apply the algorithm in Corollary 9.4.12 to po- If Po is not a tame co- 
ordinate in K[z][x,y\, then p is not a tame coordinate in K[z](x,y). If 
Po is a tame coordinate in K[z][x,y], then find a product of elementary 
automorphisms F = (po,#o) = Fi o F 2 o • • • o F n that sends x to po- 
Take the preimage G = (pi,<?i) of F under the natural isomorphism 
Aut (K[z](x,y)) — > Aut(K[z][x,y]). Then p is a tame coordinate if and 
only if pi = p. ■ 

It is also interesting to characterize effectively all coordinates (in par- 
ticular, nontame ones) in K[z\[x,y]. The following theorem is based on 
recent results of Daigle and Freudenburg [87]. Recall that a derivation 5 
of K[X] = K[x \, . . . , x n ] is called locally nilpotent if for any u G K[X] 
there exists a positive n such that 5 n (u) = 0. Following [87, Defini- 
tion 2.2], for any polynomial p = p{x , y) G X[z][x, y], we define a mapping 
A p : K[z][x,y\ ->■ K[z][x,y\ by 

A d d 

i.e., A p (f) is equal to the determinant of the Jacobian matrix of the endo- 
morphism F = (p, /) of K[z][x,y], f G #[2] [a;, y]. It is easy to see that A p 
is locally nilpotent if p is a coordinate of K(z)[x, y\. 

Theorem 9.4.14. The polynomial p = p(x , y) G AT[z][x, y] is a coordinate 
if and only if p is a coordinate in K(z)[x,y] with unimodular gradient in 
K[z][x,y\. 

Proof. Clearly, if p = p(x, y) G K[z ] [; x , y] is a coordinate of K[z\ [x,y\, then 
it is also a coordinate of K(z)[x, y], and for every automorphism F = (p, q) 
of if[z][ir,p] sending x to p, the determinant of the Jacobian matrix of F 
is a nonzero constant in K. Hence p has a unimodular gradient, and this 
gives the “easy” part of the theorem. 

Now let p G X[z][x,p] be a coordinate in K(z)[x,y] with a unimodular 
gradient in AT[z][x,p]. Restricted to the case of AT[z][x,p], Proposition 2.3 
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of [87] states that every p G K[z] [x, y ] that is a coordinate in K ( z ) [x, y] with 
a unimodular gradient in If [ 2 ] [a;, y] has the property that the derivation A p 
defined above is locally nilpotent and its kernel Ker A p is equal to If[z][p]. 
Since 



A p (x) = -Py, A p(y)=p x , 

we obtain that the ideals of K[z][x, y] generated, respectively, by the image 
of A p and by p x ,p y coincide and, by the unimodularity of the gradient, are 
equal to K[z][x,y], By the implication (3) =*> (2) of [87, Theorem 2.5], 
every locally nilpotent derivation A of if[z][#,t/] such that the ideal of 
if[z][x, 2 /] generated by the image of A coincides with If[z][x,y] satisfies 
also the condition K[z][x,y] = (Ker A)[q] for some q G K[z][x,y], Hence 

K[z][x,y\ = (Ker A)[q] = {K[z]\p])[q] = K[z]\p,q]. 

Hence the endomorphism F = (p,q) of K[z][x,y\ is an automorphism and 
p is a coordinate in K [z][x, y]. ■ 

Example 9.4.15. Let every irreducible factor of b(z) G K[z] be a divisor of 
c(z) G K[z], and let a(x , z) G K[x, z] be such that b(z) does not divide some 
coefficient di(z ), i > 0, of a x — X^ILo a i{ z ) xt • Then for any q(u , z) G K[u, z] 
with deg n q > 0, the polynomial p(x, y, z) = x + q(a(x , z) + b(z)y , z)c(z) is 
a wild coordinate in K[z][x,y], 

Proof. Applying Theorem 9.4.10, we calculate 

Vx = 1 + c(z)a x q u (a(x, z) + b(z)y, z), 
p y = b(z)c(z)q u (a(x, z) + b(z)y, z). 

It is easy to see that the first steps of a Euclidean algorithm for p x and 
p y in K[x, y , z] are the same as for a x and b(z) in K[x, z] and in this way 
we cannot reach 1 because some summand ai(z)x l of a x is not divisible 
by b(z). Now, we apply Theorem 9.4.14. Since every irreducible factor of 
b(z) divides c(z), we obtain that no solution of the equation p y = 0 in the 
algebraic closure of If is a solution of the equation p x = 0. By Hilbert’s 
Nullstellensatz, this implies that the ideal of K[x,y,z] = If[z][#,2/] gener- 
ated by p x and p y is equal to K[z][x, y]. On the other hand, p x — a x p y /b = 1 
in K(z)[x, y] and hence (see [359] or our Theorem 9.4.10) we obtain that p 
is a coordinate in K(z)[x, y]. ■ 

Corollary 9.4.16. There is an algorithm that determines whether 
a polynomial p(x,y) G If [z] [#,?/] is a coordinate. 

Proof. Given a polynomial p(x,y) G K[z][x,y], we apply the following 
procedure. 

Step 1. We choose some ordering on K[x, y , z] and calculate the reduced 
Grobner basis of the ideal of K[x, y , z] generated by p x and p y (see e.g. [13] 
for the necessary background on Grobner bases). This ideal coincides with 
K[x,y,z] if and only if its reduced Grobner basis consists of one nonzero 
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constant only. Hence, if the reduced Grobner basis obtained consists of 
a nonzero constant, then we continue with Step 2. Otherwise we conclude 
that p is not a coordinate of K [z][x, y\. 

Step 2. Working on K(z) instead of K[z], we apply the algorithm of 
Corollary 9.4.12 and determine whether p is a coordinate in K(z)[x,y\. If 
the answer is negative, then p cannot be a coordinate in jFC[^][x, y\. Other- 
wise p is a coordinate of K[z] [x,y]. ■ 

9 . 4-3 New class of wild automorphisms 

For R = K,K(z),K[z], let Aut°(i?[x,y]) be the group of augmentation- 
preserving automorphisms; i.e., F = (/i,/2) € Aut°(i?[x,y]) means that 
the polynomials /i and / 2 have no constant terms. Every automorphism G 
of R[x,y\ can be decomposed as G = T(F), where G = (x + a,y + b) for 
some a , b G R and F = (/i,/2) € Aut°(i?[x, y]). Hence, it is sufficient to 
study Aut°(i?[x,y]) instead of Aut (R[x,y]). By a result of Wright [404], 
the group T°(R[x,y]) of tame automorphisms of R[x,y] preserving the 
augmentation has a description similar to a free amalgamated product like 
Aut(AT[x, y]) does. In particular, T°(R[x, y]) is generated by the elementary 
automorphisms 

F = (z, y + f(x)), G = (x + g(y), y), 
f(x) e R[x), g(y) e R[y ], /( 0 ) = 5 ( 0 ) = 0 , 

and the diagonal automorphisms D a $ = (ax,/3y), a,/3 G R*. Since 
D a , (3 = D a j3 , 1 o Dp-ifl and Dp- 1^ is a product of elementary linear 
automorphisms (because SL2(i?) = E2(R)), we obtain that every auto- 
morphism P in T°{R[x, y]) has the form P = H\ o i/ 2 o • • • o Hk o D a , 1, 
where each Hi is an elementary automorphism. 

Every augmentation-preserving automorphism P of if[z][x,y] can be 
considered as an automorphism in Aut °(K(z)[x, y]) and has a presentation 
P = H\ o H2 o • • • o Hk o D a ^ 1, where each Hi is an elementary automor- 
phism of K(z)[x,y] and 0/aG F(z). The linear components of P and 
H10H2O' • oHk are in QL2(K[z\) and SL2 (K(z)), respectively. This easily 
implies that a € K* and both D a p and H\oH 2 o • • o Hk are automorphisms 
of AT[z][x,y]. This observation shows that, when studying automorphisms 
of if[z] [#,?/], it is sufficient to consider automorphisms that are products 
of elementary automorphisms of K(z)[x,y\. We state the following natural 
problem. 

Problem 9.4.17. Let Tk be the subgroup of Aut °(K[z][x,y]) generated 
by those automorphisms that can be presented as a product of not more 
than k elementary (and preserving the augmentation) automorphisms of 
K(z)[x,y]. Is it true that the sequence of subgroups 

Ti C T 2 C T 3 C . . . 

satisfies the ascending chain condition? 
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Obviously, every automorphism in T°(K[z][x,y]) is a product of some 
D a i and an automorphism in 7\. More generally, every automorphism of 
Jf[z][x,2/] is a product of a D a ,i and an automorphism in T^. Therefore, 
the minimal k with this property can serve as a measure of “how wild” the 
automorphism is. 

Remark 9.4.18. It is easy to see that T\ = T2. Indeed, if P = FoG G T2, 
where 

F = (x,y + f{x)), G = (x + g(y),y), f(x)eK(z)[x], g(y) e K(z)[y\, 

then obviously 

P= {x + g{y),y + f(x + g(y))) € Aut K[z][x,y], 

which means that g(y) G K[z] [y]. Hence F — PoG -1 € Aut K[z] [x, y\ and 
again we obtain that f(x) G if[z][:r]. Therefore, both F and G are tame 
automorphisms of K[z][x,y] and P G T\\ i.e., Ti = T2. 

Example 9.4.19. By [288], the automorphism of Nagata 

N = (x - 2 (y 2 + zx)y - ( y 2 + zx) 2 z, y + (y 2 + zx)z) 

considered as an automorphism of K(z)[x,y] has the presentation N = 
G _1 0F0G, where 

F = (x,y + z 2 x), G=(x + ^,y 

Since N G T3 and the Nagata automorphism is wild when considered as an 
automorphism of K[z][x, 2/], by Remark 9.4.18 we obtain that T 2 is a proper 
subgroup of X3. 

Now we start to study automorphisms in T3 of the form P = F -1 0G0F, 
where F = (x,y + f(x)) and G = (x + g{y),y) are elementary automor- 
phisms of K(z)[x,y], (Up to the order of variables, these automorphisms 
are “Nagata-like”.) Since 

P=(x + g(y + f(x)),y + f(x) - f(x + g(y + f{x)))), 

we shall also describe the corresponding coordinate polynomials x + g{y + 

/(*))• 

Lemma 9.4.20. Let f(x) G K(z)[x], g(y) G K(z)[y] be such that 
/( 0) = 3(0) = 0 and g(y + f(x)) G K[z)[x,y\. Then f(x) and g(y) have 
presentations 

/(*) = 9(y) = d{b(z)y), 

where a(x) G RT[2 ;][j;], d(y) G If [2?] [2/], a(0) = d(0) = 0, and b(z) G K[z], 
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Proof. We rewrite f(x) in the form f(x) = a(x)/b(z), where a = a(x) 6 
A"[z][x] (and a( 0 ) = 0 ), b(z ) G K[z], and a(x) = a(x,z) and b(z) are 
relatively prime in K[x, z]. In the formula 



g(y + f(x)) = g 



' , £fc ) ' \ = Y' 9 {k) (y)a k (x) 

b ( z )J ho k]bk i z ) 



G K[z\[x,y\, 



where </ k> = g^ k \y) is the A'tli derivative of g(y) (with respect to y). we 
replace x by 0 and obtain 



9{y) = g{y + f( o)) e K[z\[y\. 



Hence 



a 



g f b n 1 g n ab T 

1 ! + 2 ! 



yy g( k \y)a k {x) 

h klbk ( z ) 



G K[z\[x,y\, 



2 

- + ••• + 



g (n-l) a n-2 b 

(n-1)! 



+ 



g( n )a n ~ l 

n\ 



= 0 (mod b n ), 



and since a and b are relatively prime, we obtain that 



gtyn l g n a b n 2 o (n 1 ^a n 2 b g^a n 1 

1 - 1 \- \- 

1! 2! (n-1)! n\ 



= 0 



(mod b n ). 



Replacing x by 0 , we obtain g r b n ~ l = 0 (mod 6 n ), and hence g' = 0 
(mod 6), and the coefficient of y k in g(y) is divisible by b(z) for k > 1. 
Let g'(y) = b(z)gi(y) for some gi(y) G A"[z][t/]. Now our congruence has 
the form 



a 



'g[b n - 2 g"ab n ~ 3 g[ n ~ 2 ) a n ~ 3 b g^a ”- 2 

2 ! + 3 ! + "' + (n-1)! + n! 



=0 (mod6 n_1 ). 



Again, the coefficient of y k in gi(y) is divisible by b(z) for k > 1, and hence 
the coefficient of y k+1 in g(y) is divisible by b 2 (z) for any k > 1. Continuing 
in this way, we obtain that the coefficient of g k in g(y) is divisible by b k (z) 
for all k\ i.e., g(y) = d(b(z)y) for some d(y) G AT[z][y]. ■ 

Lemma 9 . 4 . 21 . Let a(x) G AT[2][:r], d(y) G if [2;] [2/], b(z) G K[z\, a( 0 ) = 
cf(0) = 0, and let a(x) = a(x 1 z) and b(z) be relatively prime in K[x,z]. If 
the polynomial a(x + d(a(x) + b(z)y)) — a(x) is divisible by b(z), then the 
polynomial d(y) is also divisible by b(z). 

Proof. Let b(z) = n^i&TO 2 )? where bi(z ), . . . , b s (z) are the different 
irreducible factors of b(z). It is sufficient to show that d(y) is divisi- 
ble by all polynomials b^ i = 6^(2). Since divides the polynomial 
a(x + d(a(x) 4- b(z)y)) — a(x), it divides also a(x + d(a(x))) — a(x). We 
want to show that the polynomial d(y) is also divisible by assuming 
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that a(x) is relatively prime with bi. Working modulo we obtain 
0 = a(x + d(a(x))) - a(x) = ^— d k (a(x)) (mod ft?*). 

k = 1 

Let a(x) = x q ai(x) (mod bi), a±(x) G if[z][x], and let q be the maximal 
integer with this property; i.e., ai(0) ^ 0 (mod bi). Since a(0) = 0, we 
have that q > 1. Let us first assume that d(y) is not divisible by bi. Let 
r be the maximal integer with d(y) = y r di(x) (mod bi), d\{y) G /f[z][i/], 
di( 0) ^ 0 (mod bi). Again, r > 1. For k < q, the derivative a^ k \x) has the 
form a( k \x) = x q ~ k Ck(x) (mod bi ), c k (x) G jRT[2?][x], Cfc(0) ^ 0 (mod bi). 
Hence 

aW(x)d k (a(x)) 

= x q ~ k c k (x)((x q ai(x)) r di(x q ai(x))) k = x q ~ k+kqr e k (x) (mod 6*), 

where e k (x) G If [z][x] and e k (Q) ^ 0 (mod bi). For k > q, we have that 

a^ k \x)d k (a(x)) = a^ k \(x q ai(x)) r di(x q ai(x))) k (mod bi), 

and a^ k \x)d k (a(x)) is divisible by x kqr modulo bi. If qr > 1, then the 
degrees t k of x tk dividing a^ k \x)d k (a(x)) modulo bi satisfy the inequalities 

h = q-l+qr <t 2 - q-2+2qr < • • • <t q = q 2 r < (q+l)qr < t q + i, . . . ,t n . 

Hence a(x + d(a(x))) — a(x) = x q ~ 1 ^ qr ao(x) (mod bi ), ao(x) G if[z][x], 
and ao(0) = ei(0) ^ 0 (mod bi). But this contradicts the assumption that 
a(x + d(a(x))) — a(x) = 0 (mod bi). The case qr = 1 (i.e., q — r — 1) is 
similar. Again a! (x)d(a(x)) = xc\(x) (mod bi) with c\ G if[z][x], ci(0) ^ 0 
(mod b\), and a^ k \x)d k (a{x)) is divisible by x 2 modulo bi for k > 1. Again 
a(x + d(a(x))) — a(x) = xao(x) (mod bi ), ao(x) G if[z][x], and ao(0) = 
ei(0) ^ 0 (mod bi ), which is a contradiction. Hence, in all cases we have 
that bi divides d(y). Let d(y) = b\do(y), where t > 1 and do(y) G lf[z][?/] is 
relatively prime with bi. Then the congruence 

E^^d fc (a(*)) = 0 ( mod ^) 

k = 1 

has the form 



k = 1 

and, if t < pi , we obtain a' (x)do(a(x)) = 0 (mod bi). Since a(x) is relatively 
prime with bi and a(0) = 0, we obtain that a'(x) is also relatively prime 
with bi and, hence, bi divides do(a(x)). But, as above, we see that this is 
impossible. Therefore t>pi and divides d(x). Hence b also divides d(y). 
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A derivation 5 is called triangular if S(xi) G K[xi+ i, . . . , x n ], i = 1, . . . , n. 
Clearly, every triangular derivation is locally nilpotent. For every locally 
nilpotent derivation S and for any element w in the kernel of <5, the mapping 
of K[X] 



exp (wS) = ^ 

n>0 



w n 5 n 

n\ 



is a well-defined automorphism of K[X\. 

Recall that F = (/i, . . . , f n ) of K[x i, . . . , x n \ is a stably tame automor- 
phism if its extension (/i, . . . , / n , x n +i, . . . , x n + m ) to an automorphism 
of K[x \, . . . , x n + m ] for some m > 0 is a tame automorphism of 
K[x i, . . . , x n + m ]. Similarly, one can define stably tame automorphisms of 
R[x i, . . . , x n ] for any commutative algebra R. 

The next result describes the tame and the wild automorphisms of the 
form P = F^oGoF G T 3 , where F = (x, y+f(x)) and G = (x+ g(y),y) are 
elementary automorphisms of K(z)[x, y], and shows that all automorphisms 
of this form are stably tame; i.e., they are really “Nagata-like” . Also, it turns 
out that these automorphisms are like those considered in Example 9.4.15. 

Theorem 9.4.22. Let 0 7 ^ f(x) G K(z)[x), 0 7 ^ g(y) G K(z)[y], where 
/( 0) = y(0) = 0 and f(x) = a(x)/b(z), a(x) G b(z) G K[z], 

the polynomials a and b being relatively prime in K[x,z]. Define the 
automorphisms F and G of K(z)[x,y] by 

F = (x, y + /(x)), G = (x + g(y), y). 

(i) The composition P = F _1 oGoF is an automorphism of K[z][x,y] if 
and only if g(y) = b(z)q(b(z)y) for some q(y) G AT[z][y]. In this case, 
P is equal to the automorphism exp (wS), where 6 is the triangular 
derivation of K[z][x,y] defined by 

5{x) = b(z ), S(y) = -a'(x) 
and w = q(a(x) + b(z)y) is in the kernel of S. 

(ii) Let P = F~ l o G o F be an automorphism of K[z][x,y]. Then P is 
wild (as an automorphism of K[z][x,y)) if and only if the coefficient 
of x k in f(x) is not in K[z ] for some k > 1. 

(iii) If P = F~ l o G o F G Aut K[z\[x,y], then P is stably tame and 
becomes tame as an automorphism of K[z][x,y,t] ( acting identically 
on t ). 

Proof. 

(i) Direct computations show that 

P = (Pi,P 2 ) = {x + g(y + f(x)),y + f{x) - f(x + g(y + f{x)))). 
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If P is an automorphism of K[z][x,y\, then pi,p 2 G K[z][x,y]. Applying 
Lemma 9.4.20 topi, we obtain that g(y) = d(b(z)y) for some d(y) G If [z][p] 
and b(z) G K[z). Now Lemma 9.4.21 gives d(y) = b(z)q(y ), q(y) G K[z][y\. 

Now let g(y) = b(z)q(b(z)y) for some q(y) G if[z][p]. We define the 
derivation 5 of K[z][x,y] by 5(x) = b(z), 5(y) = —a'(x). Clearly, 5 is tri- 
angular. It is easy to verify that w = q(a(x) + b(z)y) is in the kernel of 5. 
Hence exp (w5) is an automorphism of K [z][x, y] and we have to show that 
P and exp(w5) are equal on the generators x and y of if[z][x,p]. Again, 
direct calculations show that 

exp (w5)(x) = x + wSx = x + bq(a(x) + by) = pi, 

/ rw n , ^w k 5 k y 1 x~' a{k \x)b k q k {a(x) + by) 

eMwS)(y) = y + 2 —jr~ = V~t 2^ 

k > 1 * fc>l 

= V + - a(x + bq(a(x) + by)) = p 2 - 

(ii) Since P G Aut(K[z][x,y]), by (i) we obtain that g(y) = b(z)q(b(z)y) 
for some q(y) G if[^][y] and 

P - ^x + bq(a(x) + by ), y + ^(o(x) - a(x + bq(a(x) + by))) 

Hence x-hbq(a(x)+by) is a coordinate of K[z) [x, y]. If the coefficient of x k in 
f(x) is not in K[z] for some k > 1, this means that the coefficient of x k ~ x of 
a x (x , z) is not divisible by &(z), and Example 9.4.15 shows that P is a wild 
automorphism of K[z] [x, y] . Now, let the coefficients of x k in f(x) be in K[z] 
for all k > 1; i.e., f(x) = c(z)/b(z)x + x 2 /o(x,z), where c(z) and b(z) are 
relatively prime in K[z] and fo(x,z) G K[x,z]. If Q = (sc, y — x 2 /o(x, z)), 
then the automorphism P is tame if and only if Pi = Q~ l o P o Q = 
Pf 1 oGoPi is tame, where Pi = (x,y + c(z)x/b(z)). In this case, Pi is 
equal to exp (w5), where w = q(c(z)x-\-b(z)y) and the derivation 5 satisfies 
<5(x) = b(z), S(y) = —c(z). Since b(z) and c(z) are relatively prime in K[z], 
there exist b'(z ), c'(z) G K[z] such that b'c—c'b = 1; i.e., the endomorphism 

S=(x u y 1 ) = (cx + by , c'x + b'y) 

of K[z][x,y\ is a linear (and hence tame) automorphism. Now we calculate 
Pi with respect to the generators xi,pi: 

5(x i) = 0, £(yi) = c'b — b'c = —1, w = q(x i), 

Pi (S') = Pi(xi,pi) = (exp(t/;<5)(xi),exp(it;J)(2/i)) = (x 1 ,y 1 - q(x i)). 

Since Pi is elementary with respect to the tame system of generators xi, j/i, 
we obtain that Pi and hence P are tame. 

(iii) The statement follows immediately from the proof of the result of 
M. Smith [370] that for any triangular derivation 5 of K[X\ — K[x i, . . . , x n \ 
and any w G Ker 5, the automorphism exp (wS) is stably tame. If we extend 
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our S to a derivation of K[z][x,y,t] by 5(t ) = 0, we obtain that w = 
q(a(x ) + by) is still, in the kernel of 5 , 

exp (wS)(x,y,t) = (pi,p 2 ,t) = Q -1 o E o Q o E _1 , 

where Q = (x, y, t+w), E = exp (t5), and Q and E are tame automorphisms 
of K[z][x,y,t\. U 

Remark 9.4.23. Wright [404, in the proof of the corollary to Theo- 
rem 5, p. 247] gave a family of wild automorphisms of i?[x,y], where R 
is a principal ideal domain. For R = K[z] his automorphisms are given by 



P = lx + b 3 



X 2 \ 2 x 2 

y + T) 



x + b 3 




where b = b(z) G K[z] is not a constant. All these automorphisms can be 
obtained from Theorem 9.4.22 for 

/(*) = y > 9(v) = Kb) 2 - 

Theorem 9.4.22 (iii) has the following generalization, which is obtained 
by multiple applications of the idea of M. Smith [370] and gives new stably 
tame automorphisms. Note that for A = ( y , x) and the triangular automor- 
phism F = (x, y + /(x)), we have A -1 = A , and the conjugate A o F o A is 
equal to the automorphism (x + /(y), j/), which is triangular with respect 
to another ordering of the variables. 

Theorem 9.4.24. Let a* G K[x,z], bi G K[z], fi = a*/6y i = 1 
q G K[y,z] be any polynomials , and let A = (y,x) ; Fi = (x,y + fi(x)), 
i = 1, . . . ,k, be automorphisms of K(z)[x,y\. Then there exist sufficiently 
large positive integers mi, ... , mu such that for G = (x + b™ 1 . . . b™ k q(y ), y) 
the mapping 

P = Fj"" 1 o A o F^. o A o • • • o Ff 1 oGoFioAo---o F^_i o A o F& 
zs a stably tame automorphism of K[z][x, y]. 

Proof. Let q(t) = c j( z )y j f° r some polynomials cj G K[z]. Let 
i = 1, . . . , fc, be positive integers, and let ra* = r* + (n + l)sy Clearly, 

C9(») = ft* 1 E M? +(n - j>i x&w = 

j=o 

where gi(y) = £ F[z][y]. Similarly, we obtain that 

for some q^ G F[z][y]. As in the proof of Theorem 9.4.22 (iii), for k = 1 we 
fix si = 1 and obtain that 

Ff 1 o G o Fi = exp (wS), 
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where S = b 2 2 . . . b s k k 5 0 , <So(s) = ~h, S 0 (y) = (ui) x , 

w = b 2 2 ... b s k k q k (b 2 2 ...b s k k (a(x) + b ± y)) G Ker 5, 

and, after extending the action to K[z][x,y, ti], the automorphism Ff 1 o 
G o Fi is expressed in terms of triangular automorphisms 

exp(&2 2 •••b 8 k k t 1 5 0 ), {x,y,z,h + b s 2 2 . . .b s k k q k (b 2 2 . . .b s k k (a(x) + b 1 y))), 

and their inverses. Continuing in this way, we obtain that the auto- 
morphism A o F k \ o • • . o G o • • • o i o A becomes tame in F[z][X] 
for X = {xi, . . . ,Xfc+i} = {x,y, ti, . . . and is a composition of 

automorphisms of the type 

Gi = (xi + 6 ^w<(x 2 ,...,Xjb + i),a; 2 ,...,Xfc + i) 

for some polynomials Ui G if[z][x 2 , . . . ,x k +i] up to a permutation of vari- 
ables. Hence we have to show that F k l o Gi o F k is an automorphism of 
K[z][xi , . . . , Xfc+i] that becomes stably tame in K[z][x i, . . . , a;fc+ 2 ], where 
s k is big enough and F k is one of the following types (again, up to 
a permutation of variables): 

F k = (xi + a k (x 2 )/b k ,x 2 , . . . , x k+2 ), 

F k = (xi,x 2 + afc(x 3 )/6fe,X3,...,Xjb + 2), 

= (xi,x 2 + a fc (a:i)/6 fe ,X3,...,x fc+ 2). 

Straightforward calculations show that in the first two cases the composi- 
tion FjT 1 o Gi .0 F k is triangular for s k sufficiently large and hence tame. 
In the third case, we may consider F k x o Gi o F k as an automorphism 
of K(x Sl . . . , x k +i)[z][xi,x 2 \ that keeps FT[x 3 , . . . , x k +i, z][xi,x 2 \ invariant. 
As for k = 1 , we obtain that F k x o Gi o F k is a composition of tame auto- 
morphisms of K[x 3 , . . . , Xfc+i, z][xi, x 2 , x k + 2 \ , and this completes the proof. 



Theorem 9.4.22 shows that the polynomials 

p(x,y) = x + bq(a(x) + by), a(x) G K[z][x], b(z) G K[z], q(u) G K[z][u] 

are images of x under automorphisms of if[z][x,y] of a very special form. 
Now we are interested in more general polynomials than those considered 
in Example 9.4.15. We study polynomials of the form 

p(x,y) = x + d(a(x) + by ), 

where a(x) G K[z][x], b(z) G K[z], d(u) G F[z][ix], and a(0) = d(0) = 0. If 
e(z) G K[z] is a divisor of b(z) and a(x), we may replace a(x) with a\{x) = 
a(x)/e(z), b(z) with bi(z) = b(z)/e(z) J d(u) with di(u) = d(e(z)u), and 
without loss of generality we may assume that a(x) and b(z) are relatively 
prime in K[x, zj. Besides, if p(x , y) G K[z] [x, y] is a coordinate polynomial, 
then it has a unimodular gradient with respect to x and y. In our case this 
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means that 



p x = 1 + a x (x)d u (a(x) + by ), p y = bd u (a(x) + by) 

generate K[z][x,y] as an ideal. By Hilbert’s Nullstellensatz, this is equiv- 
alent to the statement that p x = p x (x,y, z) and p y = p y (x,y,z) have no 
common zeros (xo,yo,zo) h 1 the algebraic closure of K. Since a(x,z) and 
b(z) are relatively prime and a(0, z) = 0, we obtain that a x (x) is also rel- 
atively prime with b(z). If Zo is a zero of b(z ), and d(u) = d(u , z) does not 
vanish for z = zo , we derive that p x (x , y , zo) = 1 + a x (x , zo)d u (a(x 1 2 : 0 )) is 
a nonzero polynomial of x. The only case where p x and p y have no com- 
mon zeros is where this polynomial is a constant; i.e., d(u) has the form 
d(u) = e(z)u+bo(z)u 2 d\(u), where e[z) G K[z] is relatively prime with b(z) 
and bo(z) is the product of all irreducible components of b(z). Hence, with 
this exception only, we may assume that the polynomial d(u) is divisible 
by 60 ( 2 ); i- e -> it is of the form considered in Example 9.4.15. By finding an 
automorphism that sends x to p(x, y), we also shed some light on automor- 
phisms of K[z][x,y] that are products of three elementary automorphisms 

0 {K(z)[x,y]. 

Lemma 9.4.25. Let P = {pi,p 2 ) be an automorphism of K(z)[x, y] such 
that pi,p 2 £ K[z][x,y]. Then P is an automorphism of K[z][x,y] if its 
Jacobian matrix is invertible in GL 2 (K[z][x,y]). 

Proof. We make use of the Keller theorem (see [173] or [39]): If P is 
an endomorphism of K[X] that induces an automorphism of K(X) and 
the Jacobian matrix Jp of P is invertible in GL n (K[X]), then P is an 
automorphism of K[X\. In our case, since P fixes 2 , the determinant of 
the Jacobian matrix of P considered as an endomorphism of K[x,y,z] 
coincides with the determinant of the Jacobian matrix considering P as an 
endomorphism of K[z][x, y]. The proof is completed by the Keller theorem 
because P induces an automorphism of K(x, y, z). ■ 

Theorem 9.4.26. Let 0 ^ a(x) = a(x,z) £ K[z][x], 0 ^ q(u) = q(u,z) € 
K[z][u], a(0) = y(0) =0, 0 ^ b(z) G K[z], and let a(x,z) t b(z) be relatively 
prime in K[x, z\. Let bo(z) be the product of all irreducible factors ofb(z). 
Then the polynomial 

p(x, y)=x + b 0 q{a(x) + by) = x + b 0 {z)q(a{x, z) + b(z)y , z) 

is an image of x under an automorphism of K[z][x,y] that is a product of 
three elementary automorphisms of K(z)[x,y]. Moreover , p(x,y) is a wild 
coordinate if b(z) does not divide the coefficient of x % in a x (x) for some 

1 > 0 . 

Proof. We shall find a polynomial c(v) G if[^][v], c(0) = 0, such that P = 
F 2 o G o F 1 = (p, r) is an automorphism of K[z] [x, y] for some r G K[z] [x, y] , 
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where 

F 2 =^x,y+^-^, G = (x + b 0 q(by),y) 
are automorphisms of K(z)[x,y], Direct calculations show that 
P = ip(x,y),r(x,y)) 

= (x + b 0 q(a(x) + by),y + 1 {a{x) + c(x + b 0 q(a(x ) + by))) 

and the Jacobian matrix of P is equal to 

j _ ( 1 + b$q u a x boq u b \ 

{p>x H - c<y(l “h boq u ci x )) 1 -b c v boq u J 

where a x = a x (x), q u = q u (a(x) + by ), c v = c v (x + b 0 q(a(x) + by)). It is easy 
to see that the determinant of J{<p) is equal to 1. By Lemma 9.4.25, the 
proof of the theorem will be completed if we show that r(x, y) G if [ 2 ] [a?, y] 
for some c(v) G if [*][?;]. Clearly, r(x,y) is in K[z][x,y] if and only if 

a(x) + c(x + bo(z)q(a(x) + b(z)y)) = 0 (mod b(z)). 

This is equivalent to a(x) + c(x + bo(z)q(a(x))) = 0 (mod b(z)). Since b(z) 
is a divisor of ( z ) for some n, it is sufficient to show that the congruence 

a(x) + c(x + bo(z)q(a(x))) = 0 (mod b%(z)) 

has a solution c(v) G if[z][t;] for every n > 0. Clearly, the congruence has 
a solution c\(v) = —a(v) (mod bo(z)). Let us assume that we have found 
a solution c m (v) modulo b'o'(z). Then, writing c(v) in the form c(v) = 
c m (v) + b™(z)d(v ), we see that the congruence 

a(x) + c m (x+bo{z)q(a(x))) + b™(z)d(x+b 0 {z)q(a(x))) = 0 (mod b™ +1 (z)) 
has a solution d(v) G if[z][v], namely 

d ( v ) = -q^zj( a ( v ) + °m{v + bo(z)q(a(v)))) (mod b 0 (z)). 



Remark 9.4.27. The proof of Theorem 9.4.26 gives a description of the 
generators of the group T 3 defined in Problem 9.4.17. It is generated by all 
elementary automorphisms of K[z][x,y] and the automorphisms P defined 
by 

P=(p,r) = ^ x +q(a{x)+b{z)y),y+^ + -^c(x+q(a(x)+b(z)y))j , 

where a(x),b(z),c(v),d(z),q(u) are nonzero polynomials, a(x) E K[z] [r], 
b(z),d(z) E K[z\, c{v) E K[z][v], q(u) E #[z][it], a(0) = c(0) = g(0) = 0, 
such that r E K[z\[x,y\. The careful study of the proof of Theorem 9.4.26 
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gives that it is sufficient to choose in the set of generators of T 3 only one P 
for each coordinate polynomial r; i.e., by fixing r, we also fix c(v) and d(z). 

All known stably tame automorphisms of K[X] can be obtained by the 
method of M. Smith [370] and are exponential automorphisms of locally 
nilpotent derivations; see [104]. The automorphisms of Theorem 9.4.22 
and 9.4.24 are also of this kind. We do not know whether the auto- 
morphisms involved in Theorem 9.4.26 can be obtained as compositions 
(considered as automorphisms of if[^][a;,i/]) of stably tame exponential 
automorphisms of locally nilpotent derivations. 

Problem 9.4.28. 

(i) Are all automorphisms in the group T 3 C Aut A[z][x, y] stably tame? 

(ii) Are all automorphisms of K[z][x,y\ stably tame? 




10 

Test Elements of Polynomial 
and Free Associative Algebras 



Let K be a field of any characteristic, and let K[X] = K[x i, . . . , x n ) and 
K(X) = K(x i,...,x n ) be respectively the polynomial and the free as- 
sociative algebra freely generated by a set X = {xi, . . . ,x n }. The general 
problem under discussion in this chapter is how to recognize automorphisms 
of K[X ] and K(X). Following van den Essen and Shpilrain [124], we give 
the following definitions. 

Definition 10.0.1. A polynomial p = p(X) from K[X] is a test polynomial 
if for every n-tuple F 6 ( K[X]) n the equality p(F) = p(X) implies that 
the map X — >> F induces an automorphism of K[X], 

A polynomial p = p(X) is a test polynomial in a class S of polyno- 
mial mappings of K[X] if, for any p G <S, p(p) = p implies that ip is an 
automorphism of K[X]. 

Definition 10.0.2. Let S be a class of endomorphisms of K[X\ (or K(X)), 
and let M be a subclass of S. We call a polynomial p(X) — p(x i, . . . , x n ) 
a test polynomial for M in S if for any F G S the equality p(F) = p(X) 
implies F G M. If a test polynomial p is fixed by an automorphism F, we 
say that p recognizes F . 

Test elements of K(X) are defined similarly. 

Definition 10.0.3. The outer rank of a polynomial p(X) in K[X\ (or an 
element of K(X)) is the minimum number of generators Xi on which an 
automorphic image of p can depend. 
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Van den Essen and Shpilrain [124] proved that any test polynomial p(X) 
is of maximum outer rank n. 

The main purpose of this chapter is to construct test elements for 
K[X\ and K(X). This problem has been considered for various alge- 
braic systems. Dicks [93], using the description of Autlf(£i,:r2) given by 
Makar-Limanov [233] and Czerniakiewicz [86], has established that an en- 
domorphism F = (/i, /2) of the free algebra K(x i, x 2 ) is an automorphism 
if and only if [/i,/2] = a[xi,X 2 ] for some nonzero element a from K. 
Therefore the commutator [x\,X 2 ] = X 1 X 2 — X 2 X 1 is a test element for 
K(x i,z 2 ). 

It seems to be very difficult to classify all test elements (or even test 
elements in the class of monomorphisms) of polynomial and free associa- 
tive algebras. In fact, the only previously known test elements were the 

commutator [#i, X 2 ] for K(x 1, £2) [93] and x\ -\ Vx 2 n for the polynomial 

algebra R[X] given in [124]. On the other hand, it was shown in [108] that 
the exponential automorphisms of K[X] have no test polynomials. If d is 
a locally nilpotent derivation of K[X], char if = 0, and p(X) G K[X] is 
fixed by the automorphism exp 9, then there are monomorphisms of K [X] 
that are not automorphisms and also fix p(X). 

Jelonek [163] showed that a generic polynomial of degree > n in 
C[xi , . . . ,x n \ is a test polynomial for monomorphisms. The statement 
“a generic polynomial of degree d has a property W” means that there 
exists a Zariski open dense subset U C % d of the set TL d of all polynomials 
of degree d such that every element of U has the property W. 

In this chapter, we construct families of test elements for both polynomial 
algebras and free associative algebras. 

The chapter is organized as follows. In Section 10.1, we give a family of 
test polynomials for the polynomial algebra K[x 1 , . . . , x n \. In Section 10.2, 
several families of test polynomials for the free algebra K (#i, . . . , x n ) 
are constructed. In the concluding Section 10.3, we propose some open 
problems related to test polynomials. 

First we have to recall some terminology and notation. Every map 
X -» K[X\ can be extended to an endomorphism of K[X], If Xi -» /*, 
i = 1 , . . . , n, we denote the corresponding map and the induced endomor- 
phism by F = (/i,...,/ n )- In particular, we use the notation p(X) for 
a polynomial p(x 1 , . . . , x n ) and p(F) for p(/i, . . . , f n ). If F is an automor- 
phism, we call the polynomials fa coordinate polynomials. We use similar 
notation for the maps X K(X), except we call fa primitive elements. 
Recall that an automorphism F is affine if F = (ai + <71 , . . . ,a n + #n), 
where oti G K and G = (#1, . . • , <?n) is an invertible linear transforma- 
tion. Triangular automorphisms are of the form F = (/1 , . . . , / n ), where 
fa = otiXi + 0 7^ a.i G X, and the polynomials hi do not depend on 

X \ , . • • , Xi. 

Let Y = {2/1 , . . . , y n } be another set of free generators of K[X] (or of 
K(X)), and let the polynomial p(X) G K[X] be written as q(Y) = p(X). 
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If for any choice of Y £ Aut K[X] the polynomial q(Y) depends essentially 
on each variable y*, we call p(X) a polynomial of maximal rank. 

Remark 10.0.4. Let F = (or + yi, . . . , a n + g n ) be an endomorphism of 
K[X\ (or of K(X)), where G K and the polynomials gi have no constant 
terms, i = 1, . . . , n. Let H = (x\ — oq, . . . , x n — a n ). Then the composition 
of F and H is F o H = G = (yi, . . . ,y n ), and F is an automorphism 
if and only if F o H is an automorphism. Prom this point of view, it is 
sufficient to consider test polynomials in the class of all endomorphisms 
G = (yi,...,y n ) without constant terms of the polynomials y^. We call 
such endomorphisms endomorphisms without constant terms. 

Remark 10.0.5. Let F and H be automorphisms of K[X] (or of K(X)). 
If p(X) is a test polynomial recognizing F, then p(H) is a test polynomial 
recognizing G = H~ 1 (F(H)). (In this equality, we consider G and H as 
elements in (K[X]) n or (K(X)) n , not the induced automorphisms.) Clearly, 
since End (K[X]) o H = En.d(K[X]), we obtain that q(X) = p(H) is also 
a test polynomial and 

p(H)(G) = p(H(G)) = p(F(H)) = p(H). 

Hence p(H) is a test polynomial recognizing G = H~ 1 (F(H)). Obvi- 
ously, a similar statement holds for the test polynomials if we restrict 
our considerations to automorphisms and endomorphisms without constant 
terms. 

Lemma 10.0.6. If the homogeneous component of minimal degree of the 
polynomial p(X) from K[X\ (or from K(X)) is of maximal rank , then 
p(X) is a test polynomial for injective endomorphisms in the class of all 
endomorphisms without constant terms. 

Proof. Let Pk(X) be the homogeneous component of minimal degree of 
p(X), and let F = (/i, . . . , f n ) be an endomorphism from the class under 
consideration. Let G = (#i, . . . , g n ) be the linear component of F. If p(F) = 
p(X), we obtain that the homogeneous component of minimal degree of 
p(F) is Pk(G) = Pk(X). If <71, . . . , are linearly dependent, then they are 
linear combinations of yi, . . . , y n - i for a system of free generators yi, . . . , y n 
obtained from aq, . . . , x n by a linear transformation. This contradicts the 
maximal rank of Pk(X). Hence yi, . . . , g n are linearly independent, and this 
implies that the endomorphism F is injective. ■ 



10.1 Test Polynomials in Polynomial Algebras 

In this section, we assume that the ground field K is of characteristic zero 
and consider endomorphisms of K[X\ = K[x i, . . . , x n \. We start with some 
“negative” results and examples. The following example shows that the 
properties of the ground field are essential for test polynomials. 
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Example 10.1.1. If the field K of characteristic zero contains \/^T, then 
the polynomial 

p(X) = xj + --- + xl, n > 2, 

is not a test polynomial for automorphisms of K[X\ in the class of all 
endomorphisms. Indeed, let 

F= Q(l + x? + + x\ + x\),x 3 ,. . . ,x^j . 

Obviously F is not invertible because its image does not contain monomials 
of odd degree with respect to x\ and x 2 . Easy calculations show that p(F) = 
p(X)] i.e., p(X) is not a test polynomial. As we have mentioned before, p(X) 
is the only known test polynomial for R[X]. 

Example 10.1.2. Consider the Nagata automorphism [288] 

N = (x\ - 2x 2 {x\ + X 1 X 3 ) - x 3 (xl + xix 3 ) 2 , x 2 + x 3 (xl + X1X3), xs). 

It fixes q(X) = x\ + #1X3. For every polynomial u G K[xi, x 2 , X3], the 
endomorphism 

U = (x 1 - 2x 2 u - X 3 U 2 , x 2 + x 3 u, X 3 ) 

also fixes q(X). Clearly, u may be chosen in such a way that U is 
a monomorphism and not an automorphism (see [80, 108] for details). 
Changing the coordinates linearly, we obtain that q(X) = p(Y) = y\ + 
y\ + y\ over AT [a/— I]. Extending U acting identically on X3 , . . . ,# n , we 
obtain that over an algebraically closed field of characteristic zero the 
quadratic forms of maximal rank n are not test polynomials in the class of 
all monomorphisms of K[x 1 , . . . , x n ] for n > 3. 

Proposition 10.1.3. Let K be a field of characteristic zero. Then the 
polynomial x\ +x\ is a test polynomial for automorphisms of K[x i,x 2 ] in 
the class of all endomorphisms without constant terms. It distinguishes the 
orthogonal linear transformations only. 

Proof. Let L be an extension of the base field K that contains If 

p(X) = x\ + x\ is a test polynomial for automorphisms in the class of all 
endomorphisms without constant terms over L, then p{X) has the same 
property over K. Hence, without loss of generality, we may assume that 
\/— T £ K. Let F = (/1, f 2 ) be such an endomorphism and let p(F) = p(X). 
Then 

p{F) = (/1 + ~ = pPO = (*1 + - * 2 ^I) 

and we get 

/i+/2\/ z l = a{x 1 ±x 2 \f-i), O^aeK, 

and F is a linear transformation of X. Since it fixes the quadratic form 
p(X ), it is orthogonal. I 
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Proposition 10.1.4. Every homogeneous polynomial of rank 2 with degree 
d > 2 is a test polynomial of C[x 1 ,^ 2 ] for automorphisms of C[x i,£ 2 ] in 
the class of all endomorphisms without constant terms. 

Finally, we give a large class of test polynomials for K[X]. 

Theorem 10.1.5. 

(i) Let K be any field, and let . . . , k n be positive integers. Then the 
polynomial 

p{X) = X k = x k F..x k n " 

is a test polynomial for automorphisms in the class of all endomor- 
phisms of K [X] = K [x \, . . . , x n \ without constant terms. This test 
polynomial distinguishes the following automorphisms: 

F = (a i a; <T (i), . . .,a n x^ n )), 0 ^ ai, ... ,a n € K, 

where a kl . . . a kn — 1, and a is an element of the symmetric group S n 
such that — A/i j • • • , ^ . 

(ii) The polynomial 

p(x) = (x k y = (x k '...x k rr y 

with ki = n + i — l, i = 1, . . . , n, and s > 1 is a test polynomial for 
automorphisms in the class of all endomorphisms of K[X]. The only 
automorphisms distinguished by these test polynomials are 

F=(a ixi,...,a n x n ), (a kl . . . a£ n ) s = 1, a x ,...,a n eK. 

Proof. 

(i) Let F = (/i, . . . , f n ) be an endomorphism without constant terms. 
Then 

p(F)=p(X), / 1 fc ‘.../^=xf >...**» 

gives immediately that fi = aix a ^, a G S n . The end of the statement 
follows by comparison of the degrees of the variables in p(X). 

(ii) Let F = (/i,...,/„) and (f kl . . . f kn ) s = (x^ 1 . . . x kn ) s . Then all 
polynomials fi are monomials. Let 

n n n n 

oy p e k, ik-ihi^ 

j - 1 i = 1 i=lj=l 

and we obtain for the degree of Xi that 

an n + ai 2 (n + 1) + • • • + ai n (2n — 1) = n + i — 1. 

Since all are nonnegative integers and the sum of any two n + q — 1 
and n + r — 1 is bigger than 2n — 1, we obtain the only possibility, an = 1, 
aij = 0, i 7 ^ j, and this completes the proof. ■ 
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Remark 10.1.6. The statement (i) of the previous theorem is not true 
in the class of all endomorphisms. For example, the polynomial p(X) = 
x\ . . . x n is fixed by the endomorphism F = (x i . . . x n , 1, . . . , 1), which is 
not an automorphism for n > 1. 

Remark 10.1.7. Using the idea of the proof of Theorem 10.1.5 (ii), we can 
also construct test polynomials for nonlinear triangular automorphisms. For 
example, let n — 2, 

p(X) = x\(xi + a 2 (l — a 8 ) -1 #^) 3 , a E K , a ^ 0, a 8 ^ 1. 

Easy calculations show that p(X) is fixed by the automorphism 

F = (a~ 2 x i + x\, a 3 x 2 ). 

On the other hand, arguments similar to those in the proof of Theo- 
rem 10.1.5 (ii) show that p(X) is a test polynomial for automorphisms 
in the class of all endomorphisms. The automorphism F is conjugated in 
AutK[xi,X 2 \ to a linear automorphism. If we introduce a new system of 
generators 

Y = (2/1, 3/2) = (xi + a 2 (l-a 8 ) _ 1 X2,x 2 ), 

we see that F(Y) = (a~ 2 y\, a 3 ^)* On the other hand, by [108], any tri- 
angular automorphism F = (xi + hi, . . . , x n + / n ), hi E K[xi+ 1 , . . . , x n \ 
cannot be recognized by a test polynomial. 



10.2 Test Elements in Free Associative Algebras 

We start with some background on the structure and combinatorial proper- 
ties of the free associative algebra K(X) = K(x 1 , . . . , x n ) over an arbitrary 
field K . 

Proposition 10.2.1 ([80]). 

(i) The algebra K(X) is an FIR (a free ideal ring); i.e any left (right) 
ideal I is a free left (right) K(X) -module. If I is generated by a system 
of homogeneous elements, then I has a system of homogeneous free 
generators as a K(X) -module. 

(ii) If 0 ^ y h Zi e K(X), i = deg(j/i_i) < deg(yj), i = 2,...,k, 

and 1 Vi z % = 0? ^ en one of the elements yi belongs to the right 
ideal generated by yi, . . . , yi-\. 

Corollary 10.2.2. 

(i) If Ui, Vi, i = 1,2, are nonzero elements of K(X) and deg(ui) > 
deg(^i), then the equality U 1 U 2 = V\V 2 implies that u\ = v\ w, 
V 2 = WU 2 for some w E K(X). 
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(ii) //char if = 0 and uv + vu = 0 in K{X), then u = 0 or v = 0 . 

Proof. The statement of (i) follows directly from Proposition 10.2.1 (ii). 

(ii) We use induction on deg (u) + deg(v), the base of the induction 
deg(v) = 0 being trivial. Let u,v ^ 0 and deg(u) > deg(v) > 1. Since 
uv = v(—u), by (i), u = vw for some w G K(X) and (vw)v H- v(vw) = 0. 
Since K(X) has no zero divisors, we obtain that wv + vw = 0. Now 
deg(w) < deg(u), and this completes the proof. ■ 

Theorem 10.2.3. Let K be a field of characteristic 0. Then the element 
p(X) = x\ + x\ is a test element for automorphisms of the free algebra 
K(xi,X 2 ) in the class of all endomorphisms without constant terms. 

Proof. As in the proof of Proposition 10.1.3, we assume that y/^1 G K. 
Let F = (/i 5 fyf) be an endomorphism without constant terms and such 
that p(F) = p(X). Let 

fi = 9i + 92 + h 9k i /2 = hi + /12 + h hk , 

where gi and hi are the homogeneous components of degree i of /i and 
/ 2 , respectively. If hk ^ 0 and k > 1 , by comparing the homogeneous 
components of degree 2k of p(F) and p(X), we obtain that g\ + h\ = 0; 
i.e., gk = hkV~ I- Now we apply inductive argument. We assume that 
for some i > 1 we have established gj = hj^f-l, j = i + 1, . . . , k. The 
comparison of the homogeneous components of degree k + i gives 

k 

^ ^X ds9i-\ -k—s H - h s hi+k— s ) = 0 , 
s=i 

{hi + giy/—l)hk + hk(hi + giV—1) = 0 . 

By Corollary 10.2.2 (ii), gi = In this way we establish that g\ — 

The homogeneous components of degree 2 of p{F) and p{X) satisfy 
the equality g\ + h\ = x\ + x\, which contradicts g\ = /ii\/— 1. Therefore 
the map F is linear. Since it leaves invariant the quadratic form x\ + 
it is invertible, and this means that F is an automorphism. ■ 

Now we construct a family of multilinear test polynomials recognizing 
automorphisms in the class of endomorphisms without constant terms. 

Theorem 10.2.4. Let K be a field of characteristic 0 and let 

p{X) = ^ ^ ^a-^cr(l) • • • %a(n) ^ ^*(-^0 = K{X \ , . . . , X n ) , 

<r€5 n 

where a G G K and S n is the symmetric group of degree n. IfJ^aes ^ 0, 
then p(X) is a test element that recognizes automorphisms in the class of 
all endomorphisms without constant terms. This test element distinguishes 
linear automorphisms only. 
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Proof. Let F = (/i, . . . , f n ) be an endomorphism without constant terms 
such that p(F) = p{X). First we assume that F is not linear. Let gi be the 
homogeneous component of fi of maximal degree. Since the element p(X) 
is multilinear, the homogeneous component of maximal possible degree of 
p(F) = p(/i, . . . , fn) is equal to p(G) = p(gi , . . . , g n ). Since F is not linear, 
deg(^) > 1 for some i, and we obtain 

P(G) = p{gi , • • • , g n ) = ^2 a °9*(i) • • • 9cr(n) = 0. 

creS n 

Hence, for any r E 5 n , 

P{t{G)) = p{g T (l), ■ • • ,5r(n)) = ^2 a <r9*r( 1) • • • 9<TT(n ) = 0. 

*€S n 

We make a summation of these equalities allowing r to run through S n 
and, since S n r = 5 n , we obtain 

q( G ) = ( a p) ( ^c 1 ) • • -^(«) ) = °- 

v pes n / ^aes n / 

Therefore, we may assume that in the equality p(G) = 0 all coefficients a a 
are equal to 1; i.e., 

P{ X) ^ ^ ^'<t( 1) • • • %cr{n)' 
cr£:S n 

Let hi be the leading term of gi with respect to the lexicographical or- 
der. Then the leading term of every product g a ^ . . . av( n ) in p(G) is 
h a ( i) . . . h a (n) and the leading term of p(G) is a sum of some h a ( i) . . . h^n). 
Since the base field is of characteristic 0, we obtain that the leading term 
of p{G) is different from 0, which is a contradiction. In this way we es- 
tablish that all polynomials fi are linear. Now the map F induces a map 
F = (/i, . . . , f n ) of the polynomial algebra K[X) and 

p(X)=p(F) = n\f 1 ...f n . 

By using Theorem 10.1.5, we see that the polynomials in commuting vari- 
ables fi are linearly independent. Since fi are linear, this implies that fi 
are also linearly independent; i.e., F is a linear automorphism of K(X). 
This completes the proof of the theorem. ■ 

Remark 10.2.5. As in the commutative case, the statement of Theo- 
rem 10.2.4 does not hold if we consider the class of all endomorphisms of 
the free algebra. 

Corollary 10.2.6. Let K be a field of characteristic 0, and let 

p(A) — ^ ] &<T%cr(l) • • • ^ -^"(-^0 X(x \ , . . . , X n ) , 

creSn 
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where a a G K, S n is the symmetric group of degree n, and ^ 0* 

For any fixed s > 1, the element 

q (X)=p(xt n ,x f +1) ,...,< (2n_1) ) 

is a test element that recognizes automorphisms in the class of all 
endomorphisms. 



Proof. Let F be an endomorphism of K(X) such that q(F ) = q(X) and 

let G = (<7i, . . . ,#n)j where gi = Let hi be the homogeneous 

component of maximum degree of gi. If F is not affine, as in the proof of 
Theorem 10.2.4, we obtain p(hi , . . . , h n ) = 0, which is impossible. Hence 
F is affine. Now we consider the endomorphism F of K[X] induced by F. 
Clearly, 



q(X)=q(F)=p(G) = 



( ^ ^<7 j 9l • • • 9n 

' *es n ' 




jsn js{n-\-l) 


js(2n— 1) 


-sn-s(n+l) 

«^2 


-s(2n— 1) 

. . dj n 



Applying Theorem 10.1.5 again, we obtain that F is a linear automorphism 
of K[X], and the same holds for F in K(X). ■ 

Up to conjugation, most of the examples of test polynomials in the 
commutative case recognize linear automorphisms only. Now we construct 
a family of automorphisms of K(X) that recognize larger classes of au- 
tomorphisms. The construction is based on the result of Dicks [93] that 
the commutator [xi,^] is a test polynomial that, up to a multiplica- 
tive constant, distinguishes automorphisms of K(xi,X 2 ) in the class of all 
endomorphisms . 

Lemma 10.2.7. Let n > 2 and let every monomial of h\,h<i G K(X) 
depend essentially on some of the variables X 3 , . . . ,x n . Then the equality 

[xi + h 1 ,x 2 + h 2 \ = [* 1 , 0 : 2 ] 



implies hi = h 2 = 0 . 

Proof. Without loss of generality, we may assume that hi ^ 0. Let W C 
K (X) be the vector space spanned on all monomials that depend on some 
X 3 , . . . , x n . Every monomial from W can be written in the form 

w = wi(xi 1 x 2 )x il . . .Xi k w 2 (xi,x 2 ), k > 1 , 

where Wi and w<i depend on x\ and X 2 only and both the indices ii and ik 
are different from 1 and 2. Obviously, W is a free K(xi, X 2 )-bimodule with 
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basis consisting of all monomials 

Xi 1 ...Xi k , k > 1, 

Let p be the minimal degree of the nonzero homogeneous components of 
h\ and h 2 , and let u\ and u 2 be the homogeneous components of degree p 
of hi and /12, respectively. Clearly, 

[xi , U2] + [ui , X2] = X1U2 — X2U1 - u 2 x 1 + uix 2 — 0. 

We express ui and U2 in the form 

«1 ^^ 2 s U (Xi,X 2 )Xi 1 . . .Xi k S2i(xi,X 2 ), 

Vj 2 = ^ ^ ^2i(*El) ^2)5 

where the summation runs over all ii, , i k with h,ik ^ 1 , 2 . Therefore, 

[® 1 ,« 2 ] + [ui,X 2 ] - . ..X ik S 2i - X 2 t li Xi 1 . ,.X ik t 2i 

• • • Xi k S2iXi “l - tnX{ ^ . . . Xi k t 2 iX2) = 0. 

Since W is a free K(xi,X2)-bimodule, for any fixed (ii, . . . ,ik) we obtain 
the following equality in K(x ux 2 ) 0 x X(xi,x 2 ): 

XlSu 0 s 2 i - x 2 tu 0 t 2 i - Su 0 S 2 iXi + tu 0 1 2 ^2 = 0. 

By comparing the homogeneous components of minimum degree of s 2 i and 
t2i, we obtain that they are 0 , which is a contradiction. This completes the 
proof of the lemma. ■ 

Theorem 10 . 2 . 8 . Let ii , . . . ,i r ,ji, • • • ,jr be a set of indices such that 
h ^jk,k=l,...,r, and 

{h,jk \k=l,...,r} = {1, . . . ,n}. 

Then the polynomial 

p(X) = [x il ,x jl }...[x ir ,x jr ] 

is a test polynomial recognizing automorphisms in the class of all 
endomorphisms of K(X). 

Proof. Let F = (/i,...,/ n ) be an endomorphism of K{X) such that 
p(F) = p(X). Since the commutators [fi k ,fj k \ have no constant and linear 
terms and deg(p(X)) = 2 r, we obtain 

IfikJjk) = 0 ^ a k e K. 

For example, let [/i,/2] = a[xi,x 2 ]. We write fi and /2 in the form 

h = 9 i(xi,x 2 ) + h u f 2 = g 2 (xi,x 2 ) + h 2 , 

where each monomial of hi, /12 depends on some of the variables £3, . . . , x n . 
Therefore, 

[gi(xux 2 ),g 2 (x u x 2 )] = a[xi,x 2 ] 
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and, by the result of Dicks [ 93 ], (^1,^2) is an automorphism of K(x 1,0:2). 
We extend it to an automorphism G = (<71,02? # 3 ? ••• ,x n ) °f K(X). 
Let G -1 = (ui,U2,X3, . . . ,x n ) be the inverse of G. Clearly, [1/1,112] = 
a~ x [x 1,^2] and the composition of automorphisms V = F o G _1 has the 
property [^1,^2] = [#i?# 2 ]- On the other hand, it is easy to see that 



Vi=Xi+ W i,i>2 = x 2 4 - W2, 



where each monomial of Wi,W2 depends on some £3, . . . ,x n . By Lemma 
10 . 2 . 7 , this implies that w\ = w 2 = 0 ; i.e., /1 = fi(xi,x 2 ), / 2 = f2{xi,x 2 ), 
and (/1 , /2) is an automorphism of if (#i , X2 ) . In this way, we obtain that for 
each pair of indices i k ,j k , the pair = ( fi k ,fj k ) belongs to (K(x ik ,x jk }) 2 

and induces an automorphism of K (xi k , Xj k ) . 

Now, if any index i appears exactly once in the indices ii,..., 7 r , 
we obtain that F is decomposed as a product of automor- 
phisms G k such that each G k fixes all x s different from Xi k and Xj k and 
fikifjk € if (%i k i x jk)‘ Therefore, F is an automorphism. If, for example, 
the index 1 appears twice, say as i\ and %2 , then f\ 6 if(#i, Xj x ) flif (x \ , Xj 2 ) 
and fi = Since (/1,/jJ is an automorphism of K(xi,Xj 1 ), we ob- 

tain that /1 = £ + ryxi, £, rj € if , rj ^ 0 , and fi is a linear polynomial in #i. 
Now p(X) is presented as a product of three kinds of commutators: 

(i) [a;i,a;j], where the variables Xi and xj appear in other commutators 
as well; 

(ii) [x{, x k \, where x\ appears in other commutators, whereas x k does not; 

(iii) [xi,x m ], where neither xi nor x m appear in other commutators. 

As a result of our observations, we obtain that 

ViXii fj — “f~ VjXj ? £,i-> <v? £ if? 0 7 ^ Vi iVj ^ ^ 1 

fk = VkXk+9k(xi), O^T/fcGif, g k eK(xi ), 

(. fljm ) = (fl(xi,x m ),f m (xi,x m )) eAutK(xi,X m )- 

Again, F is a composition of automorphisms that fix all but one or two of 
the variables, and this implies that F itself is an automorphism. ■ 

The following corollary is an analog of a result of Zieschang [ 417 ] in the 
case of free groups. 

Corollary 10 . 2 . 9 . The element 

p(X) = [*1, x 2 \ ■ ■ ■ [x 2n -i, X 2n \ 

is a test element in the class of all endomorphisms of K (xi , . . . ,#2n)* 
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10.3 Open Problems 

Problem 10.3.1. Is the element 

p{X) = xj + --- + xl, X = (x 1 ,...,x n ), n> 2, 

a test element for automorphisms in the class of endomorphisms of 
K(X) without constant terms over any field of characteristic 0? By 
Example 10.1.2, this is not true for K[X ] when K is algebraically closed. 

Problem 10.3.2. Let X = {x ±, . . . , # 2 n}, 

U = [xi,* 2 ] + [*3,3:4] + h [X 2 n-l,x 2n ]. 

Is u a test element of the free associative algebra K(X)? 

Problem 10.3.3. Let F be an automorphism of K[X], char K = 0, and 
let p(X) be a test polynomial for F. Is F conjugate to a diagonalizable 
linear automorphism? 

Problem 10.3.4. Let F = (xi + hi , . . . ,x n 4- h n ) be a triangular auto- 
morphism of K{X), where the elements hi do not depend on x\ 
and let F be nonaffine; i.e., some of the elements hi are of degree > 2. Is 
there a test element recognizing F in the class of all endomorphisms with 
or without constant terms? By [108], in the commutative case, F has no 
test polynomials. 

Problem 10.3.5. Describe linear automorphisms that possess test 
polynomials. Is it true that a diagonal automorphism 

G = (£l#l, • • . , £n^n)> i = 1, . . . , 71) 

has a test polynomial if and only if there exist positive integers fci , . . . , k n 
such that 

#*...£» = 1? 

Conjecture 10.3.6. For n > 2, the standard polynomial 

s n (X) = s n {xi,...,x n ) = ^ (signer)*^!) ...x a{n) 

(rS n 

is a test element for K(X), X = {#i, . . . , x n }. It has the property that for 
an endomorphism F of K(X), 

s n (F) = os n (X), 0 ^ a G K 

if and only if F is linear and invertible for n odd and F is affine and 
invertible for n even, n > 2. 

Conjecture 10.3.7. Any nontrivial multilinear polynomial of degree n is 
a test element for automorphisms of K{X), X = {xi, . . . , x n } in the class 
of all endomorphisms without constant terms. 




10.3. Open Problems 181 



Conjecture 10.3.8. Any nontrivial multihomogeneous polynomial p(X) 
that depends essentially on all variables X is a test polynomial for auto- 
morphisms of K(X), X = {^i, . . . ,x n } in the class of all endomorphisms 
without constant terms. If deg x .(p) = n + i — 1 , i = 1 , . . . , n, then p(X) is 
a test element in the class of all endomorphisms. 

Problem 10.3.9. Is it true that for any automorphism F of K(X) (or for 
any tame automorphism) there exists a test element p(X) (in some class 
of endomorphisms) such that 

p(F) = ap(X), O^aeK? 

For example, is the element 

p(x 1 ,x 2 ,x 3 ) = [xufi(x2,X3)][x2,X 3 ], /( 0,0) ^ 0 

a test element recognizing F = (x i + fi(x2 , £3), X2, x 3 )l 

A polynomial P G C[X] of degree d is called a general polynomial in C[X] 

if the coefficients of all monomials x] 1 . . . x\ with 0 < i\-\ 1- i n < d 

in 

p = Yj a h-i n x 

0<iiH \-i n 

are nonzero and algebraically independent over Q. 

Conjecture 10.3.10. Let P € C[xi,...,x n ] be a general polynomial of 
degree d > n > 2 . Then P is a test polynomial recognizing only the identical 
automorphism. 
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In this part of the book, we study free algebras of Schreier varieties of 
algebras. 

Let K be a field. A variety 9 71 of algebras over K is defined as a class of 
algebras closed under taking subalgebras, homomorphic images, and direct 
products. Let A be a set. An algebra F<m(X) in 97t is a free algebra with 
the set X of free generators if F%i(X) is generated by A, and if / is an 
arbitrary map from X to any algebra A of 9 7t, then / can be extended to 
a homomorphism F<m(X) A (it is clear that such extension is unique). 
The algebra Fgjj(A) is determined by X and 97t uniquely up to an isomor- 
phism. The cardinality of X is called the rank of F<m(X). For any nontrivial 
variety 9 7t and a set A, there exists the free algebra Fgji(X). If A G 97t, 
then there exists a set A such that A is a homomorphic image of F%i( A). 
In particular, A is isomorphic to a factor algebra of Fgjt( A). 

Let A be a finite set, A = {aq, . . . ,x n }. A pair (u,v) of elements of 
Fgji(X) is called an identity in an algebra A of 9Jt if u(ai, . . . ,a n ) = 
v(ai, . . . , a n ) for all elements ai, . . . , a n of A. The following result is due 
to G. Birkhoff. Let 97t be a variety of algebras. A subclass 9t in 97t is 
a (sub)variety if and only if there exists a set of identities (ui,Vi), i G /, 
such that consists of all algebras of 97 X satisfying these identities. Here 
Ui,Vi ( i G I) are elements of F$n with a countable set A of free genera- 
tors. A variety of algebras is called homogeneous if it is defined by a set of 
homogeneous identities. A variety of algebras is said to be Schreier if any 
subalgebra of a free algebra of this variety is free in the same variety of 
algebras. 
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Chapter 11 considers basic properties of free algebras of homogeneous 
Schreier varieties of algebras. In Section 11.1, we describe the main types 
of Schreier varieties and introduce universal multiplicative enveloping al- 
gebras of free algebras. In Section 11.2, we expose the weak algorithm for 
free associative algebras and discuss Schreier’s techniques for free algebras: 
ranks of left ideals of free associative algebras and Schreier-type formulas 
for ranks of subalgebras of free algebras. 

Using free differential calculus, we introduce partial derivatives of ele- 
ments of free algebras in Section 11.3. We show that free algebras of the 
main types of Schreier varieties have the property of differential separabil- 
ity for subalgebras. At the same time, free Lie superalgebras are residually 
finite. Theorems 11.3.6-11.3.8 give some necessary and sufficient conditions 
for a homogeneous variety of algebras to be Schreier. Theorem 11.3.11 gives 
an effective algorithm to find ranks of subalgebras of free algebras. In Sec- 
tion 11.4, we consider equivalence and stable equivalence of elements under 
the action of the automorphism group. Section 11.5 shows that an injective 
endomorphism of maximal rank does not change the rank of a system of 
elements of maximal rank. 

In Chapter 12, we consider automorphic orbits and combinatorial prop- 
erties of primitive elements in free algebras of the main types of Schreier 
varieties of algebras. A system of elements of a free algebra F is primitive 
if it is a subset of some set of free generators of F. The rank of a system 
of elements of a free algebra F is the minimal number of generators from 
X on which automorphic images of these elements can depend. 

Let A be a finitely generated free algebra of the variety QJlo (the variety 
of all algebras), 9Jti (the variety of all commutative algebras, char AT ^ 2), 
or 9112 (the variety of all anticommutative algebras). 

In Sections 12.1 and 12.2, we consider basic properties of partial 
derivatives of elements of A. Section 12.3 is devoted to elimination of 
variables. 

In Section 12.4, we obtain the rank theorems. Theorems 12.4.7 and 12.4.8 
show that the rank of a system of elements is equal to the rank of the free 
module generated by the elements whose components are partial deriva- 
tives of elements of this system. Theorems 12.4.9 and 12.4.10 are the rank 
theorems for free Lie algebras and superalgebras. 

In Section 12.5, we study primitive systems of elements and actions of 
endomorphisms on automorphic orbits of elements. Theorem 12.5.1 gives 
a matrix characterization of primitive systems of elements: a system of 
elements of A is primitive if and only if the matrix of partial deriva- 
tives of elements of the system is left invertible over U(A). Based on 
Theorems 12.4.7-12.5.1, we obtain in Theorem 12.5.2 algorithms to find 
the rank of a system of elements and to decide whether this system is 
primitive or not. Theorems 12.5.3-12.5.8 give matrix characterizations of 
primitive systems of elements in free Lie algebras and super algebras. Using 
the Freiheitssatz, we obtain another characterization of primitive elements 
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in Theorem 12.5.15: an element a of A is primitive if and only if the factor 
algebra of A by the ideal generated by a is a free algebra in the same variety 
of algebras. 

In Section 12.6, we consider actions of endomorphisms on primitive 
elements: endomorphisms of A preserving primitivity of elements are 
automorphisms (this statement is also true for free Lie algebras and su- 
peralgebras). Using this result, we obtain a more general statement in 
Section 13.3 of Chapter 13. Theorem 13.3.3 says that if an endomorphism 
of A preserves the automorphic orbit of a nonzero element, then it is 
an automorphism of A. In Theorems 12.6.6-12.6.8 and Problems 12.6.9 
and 12.6.10, we consider images of nonzero linear combinations of free 
generators. 

In Chapter 13, we consider fixed points of endomorphisms and automor- 
phisms of free algebras and generalized primitive elements. Let K be a field, 
charier ^ 2, and let X be a finite set, X = {xi, . . . ,x n }, F = F(X) the 
free F-algebra without the unity element on the set X of free generators 
of one of the following varieties of algebras over a field K : the variety of all 
algebras, the variety of Lie algebras, varieties of color Lie superalgebras, 
the variety of Lie p-algebras, varieties of color Lie p-superalgebras, and 
varieties of commutative and anticommutative algebras. 

A subalgebra H of F is said to be a retract if there is a homomorphism 
p: F H such that p is the identity mapping on H (p is the retraction 
homomorphism). Obviously, a subalgebra H of F is a retract if and only if 
there exists an ideal I of F such that F = H 0 /. 

In Section 13.1, we consider test elements of the algebra F = F(X): 
we prove that test elements for monomorphisms of the algebra F are the 
elements of the maximal rank (Theorem 13.1.1); test elements for endo- 
morphisms of F are elements that do not belong to any proper retract 
of F (Theorem 13.1.4). Section 13.2 characterizes retracts of the algebra F 
(Theorem 13.2.1). In Section 13.3, we prove that if an endomorphism cp of 
the algebra F preserves the automorphic orbit of a nonzero element of F, 
then actually ip is an automorphism (Theorems 13.3.1-13.3.3). 

After that, we study generalized primitive elements of free algebras and 
their automorphic orbits. An almost primitive element (APE) of a free alge- 
bra F is an element that is not primitive in F but is primitive in any proper 
subalgebra of F containing it. Section 13.4 deals with almost primitive and 
test elements of free algebras with the Nielsen-Schreier property. We show 
that almost primitive elements are elements of maximal rank. If the free 
algebra F is a free product of its subalgebras A and B , and elements a 
and b are homogeneous almost primitive elements in algebras A and B, 
respectively, then the element a + b is an almost primitive element in F. 
This result gives us a possibility to construct in Theorem 13.4.7 a series of 
almost primitive elements. We discuss also a connection between properties 
of elements to be almost primitive and to be test elements. 
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By A we denote the left ideal of the free associative algebra K(X) gen- 
erated by the set X. Following [356], we say that an element u of L(X) 
is a A-primitive element if the elements -^ru, . . . , -^-u generate A as the 
left ideal of K(X). 

In Section 13.5, we prove Theorem 13.5.1: If the rank n of the free Lie 
algebra L is even, n = 2m, then any A-primitive element of L is an au- 

tomorphic image of [x\,x^ + b [x 2 m-u% 2 m\, there are no A-primitive 

elements of L if n is odd, and the automorphism group of the algebra L 
acts transitively on the set of all A-primitive elements. 

In Section 13.6, we consider generic elements of free Lie algebras. Let il 
be a variety of Lie algebras over a field K defined by the set of identities 
23, L = L(X) the free Lie algebra with the set X of free generators, and 
let 23(L) be the verbal ideal of L defined by 23. An element / of L is said 
to be il-generic if / G 23(L) but / ^ 23(37) for every proper subalgebra H 
of L. Let 21 be the variety of abelian Lie algebras. An element u of the free 
Lie algebra L is 2l-generic if u e [L, L ], and if u G [if, if] for some subalge- 
bra H of L, then if = L. Propositions 13.6.1 and 13.6.3 give new examples 
of almost primitive elements of free Lie algebras. In Proposition 13.6.4 and 
Theorem 13.6.5, we construct series of examples of 2l-generic elements of 
free Lie algebras. Theorem 13.6.7 gives an example of an 2l-generic element 
of the free Lie algebra L(x,y) that is not an APE of this algebra. In The- 
orem 13.6.8, we show that the element u = [xi,...,x n \ is an 9t n -generic 
element of the free Lie algebra L(X), where 9t n is the variety of nilpotent 
Lie algebras of degree not less than n. 

Chapter 14 considers free Leibniz algebras and their automorphisms. Let 
K be a field and L a AT-algebra with the bracket multiplication [ , ] . Then 
L is a Leibniz algebra if 

[x,[y,z]] = [[x,y],z] - [[x,z],y] 



for all x,y,z E L. 

The variety of Leibniz algebras is not a Schreier variety of algebras. 
Nevertheless, many combinatorial properties of this variety are similar to 
the properties of Schreier varieties of algebras. 

In Section 14.2, we prove that the variety of all Leibniz algebras has 
the property of differential separability for subalgebras, Theorem 14.2.1. 
Theorem 14.2.2 shows that the Jacobian conjecture has a positive solution 
for free Leibniz algebras. 

In Section 14.3, we study finitely generated subalgebras of the free Leib- 
niz algebra. An algebra R is called residually finite if for any element a E R 
and for any finitely generated subalgebra B such that a £ B there exist 
a finite-dimensional algebra H and a homomorphism <p: R—> H such that 
(p(a) £ <p{B). Theorem 14.3.8 shows that free Leibniz algebras are residu- 
ally finite. It follows that the occurrence problem for free Leibniz algebras 
is algorithmically decidable. 
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In order to clarify differences between the variety of Leibniz algebras 
and the Schreier variety of algebras, we consider automorphisms of free 
Leibniz algebras in Section 14.4. Automorphisms of finitely generated 
free algebras of Schreier varieties of algebras are tame; that is, they are 
compositions of elementary automorphisms (see Section 11.1 of Chap- 
ter 11). In Section 14.4.1, Theorem 14.4.5 gives a characterization of 
tame automorphisms. Section 14.4.2 collects corollaries of the main re- 
sult. Theorem 14.4.7 shows that there is an algorithm to recognize tame 
automorphisms. Theorem 14.4.8 gives an example of a wild automorphism. 
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Schreier Varieties of Algebras 



In this chapter, we consider the main properties of free algebras of Schreier 
varieties of algebras. A variety of algebras is said to be Schreier if any 
subalgebra of a free algebra of this variety is free in the same variety of 
algebras. In Section 11.1, we describe the main types of Schreier varieties 
and introduce universal multiplicative enveloping algebras of free algebras. 
Theorem 11.1.1 gives the main properties of the free algebras of these va- 
rieties. In Section 11.2, we expose the weak algorithm for free associative 
algebras and discuss Schreier’s techniques for free algebras: ranks of left 
ideals of free associative algebras and Schreier-type formulas for ranks of 
subalgebras of free algebras. 

Using free differential calculus, we introduce partial derivatives of el- 
ements of free algebras in Section 11.3. Theorem 11.3.1 shows that free 
algebras of the main types of Schreier varieties have the property of dif- 
ferential separability for subalgebras. Using the Poincare-Birkhoff-Witt 
theorems, we prove in Theorem 11.3.4 that Lie super algebras are differ- 
entially separable. Theorem 11.3.5 shows that free Lie superalgebras are 
residually finite. Theorems 11.3.6-11.3.8 give some necessary and sufficient 
conditions for a homogeneous variety of algebras to be Schreier. Theo- 
rem 11.3.11 gives an effective algorithm to find ranks of subalgebras of free 
algebras. In Section 11.4, we consider equivalence and stable equivalence 
of elements under the action of the automorphism group (Theorems 11.4.4 
and 11.4.5). 

Theorems 11.5.3 and 11.5.4 of Section 11.5 show that an injective en- 
domorphism of maximal rank does not change the rank of a system of 
elements of maximal rank. 
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11.1 Main Types of Nielsen-Schreier Algebras 

Let K be a field. A variety 9 K of algebras over K is a class of algebras 
closed under taking subalgebras, homomorphic images, and direct prod- 
ucts. Let I be a set. An algebra F^i{X) in 9Jt is called the free algebra 
with the set X of free generators if F$n(X) is generated by X, and if / 
is an arbitrary map from X to any algebra A of 9Jt, then / can be ex- 
tended to a homomorphism F$ji(X) -> A (it is clear that such extension is 
unique). The algebra Fgjt(X) is determined by X and 9Jt uniquely up to an 
isomorphism. The cardinality of X is called the rank of Fgyi(X). For any 
nontrivial variety DJI and a set X there exists the free algebra Fgji(X). If 
A G 9Jt, then there exists a set X such that A is a homomorphic image of 
Fgjt(X). In particular, A is isomorphic to a factor algebra of Fgji(X). 

Let X be a finite set, X = {xi, . . . ,x n }. A pair (u,v) of elements of 
Fgrrt(X) is an identity in an algebra A of DJI if u(ai, . . . , a n ) = u(ai, . . . , a n ) 
for all elements ai, . . . , a n of A. The following result is due to G. Birkhoff. 
Let SOT be a variety of algebras. A subclass 9t in 9Jt is a (sub)variety if and 
only if there exists a set of identities (u il Vi), i G /, such that 91 consists of 
all algebras of 9Jt satisfying these identities. Here Vi (i G I) are elements 
of Fgjt with a countable set X of free generators. A variety of algebras is 
called homogeneous if it is defined by a set of homogeneous identities. For 
more details about varieties of algebras, we refer to [25, 79, 143, 178]. 

A variety of algebras is said to be Schreier if any subalgebra of a free alge- 
bra of this variety is free in the same variety of algebras; this notion came 
from group theory. In the 1920s, J. Nielsen [294] and O. Schreier [329] 
proved that any subgroup of a free group is free. A. G. Kurosh proved 
in [195] that subalgebras of free nonassociative algebras are free. A. I. Shir- 
shov [345] showed that the variety of Lie algebras is Schreier (this result 
was also obtained by E. Witt in [403], and he also proved that the variety of 
all Lie p-algebras is Schreier). A. G. Kurosh in [197] extended this result to 
the variety of all fl-algebras. An fi-algebra is a vector space with a family 
Q of basic multilinear multiplications of varieties at least 2. In this paper, 
he also classified subalgebras of free products of f2-algebras. Earlier, for the 
case of all nonassociative algebras it was done in [196], and for the case 
of all commutative and all anticommutative algebras it was done in [139]. 
Yu. A. Bahturin [30, 31] and M. V. Zaicev [414] proved that Schreier va- 
rieties of Lie algebras are the variety of all Lie algebras and the variety 
of abelian Lie algebras, respectively. A. I. Shirshov showed in [346] that 
subalgebras of free nonassociative commutative and anticommutative alge- 
bras are free; that is, the varieties of commutative and anticommutative 
algebras are Schreier. 

A. A. Mikhalev in [243] and A. S. Shtern in [369] showed that the variety 
of Lie superalgebras is Schreier. A. A. Mikhalev proved this result for the 
variety of color Lie p-superalgebras in [245] . Up to now there is no complete 
classification of Schreier varieties of algebras. U. U. Umirbaev [388, 392] 
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obtained new examples of Schreier varieties of algebras and gave necessary 
and sufficient conditions for a variety of algebras to be Schreier. As for 
subalgebras of free algebras of varieties of linear fi-algebras, see articles 
[26, 38]. 

Let K be a field, X a set, and let F = F(X) be the free algebra of 
a homogeneous variety of algebras with the set X of free generators over 
the field K. For u E F(X), by £(u) = £x(u) we denote the degree of u. 
Consider also a generalized degree function fi: X -» N, where N is the 
set of positive integers. Let T(X) be the free groupoid of nonassociative 
monomials in the alphabet X, S(X) the free semigroup of associative words 
in X, and ~ : r(X) — > S(X) the bracket removing homomorphism. We set 
jjl(x i . . . x n ) = Y^=i vfai) f° r x i € If /i(x) = 1 for all x E X, then (i 
is just the usual degree, /i = £. If a E F, a = J2 a i a i, 0 ^ a* E AT, a* are 
basic monomials, aj ^ a s with j ^ s, then we set fi{a) = ma x*{/x(a*)}. By 
a° we denote the leading part of a: a° = Yhj^{ aj )=^{ a ) a j a j- 

A subset M of F is called independent if M is a set of free generators 
of the subalgebra of F generated by M. A subset M = {a*} of nonzero 
elements of F is called reduced if for any i the leading part a° of the 
element a* does not belong to the subalgebra of F generated by the set 
{a°j | j ± i}- 

Let S = {s a | a E 1} be a subset of F. A mapping u: S S' C A 
is an elementary transformation of S if either a; is a nonsingular linear 
transformation of S', or cu(s a ) = s a for all a E /, a ^ /?, and w(sp) = sp + 
f({s a | a 7 ^ /?}), where / is an element of a free algebra of the same variety 
of algebras. It is clear that elementary transformations of free generating 
sets induce automorphisms of the algebra F; such automorphisms are called 
elementary. 

Let A be a free algebra of a homogeneous Schreier variety of algebras. 
One can transform any finite set of elements of the algebra A to a reduced 
set by using a finite number of elementary transformations and possibly 
cancelling zero elements. Every reduced subset of the algebra A is an in- 
dependent subset (this is what is called the Nielsen property). Moreover, 
by using Kurosh’s method, one can construct a reduced set of generators 
for any subalgebra of the free algebra A. Indeed, let B be a subalgebra 
of A, M = Ujlo Mj, where Mj consists of G-homogeneous elements of de- 
gree j, if nonempty. We set Mo = 0. Then, by induction, we suppose that 
Mp, . . . , Mi have been defined. Let Bi be the subalgebra of B generated by 
[fj=oMj and Si = {b E Bi \ £(b) < i + 1}. Consider the G-homogeneous 
subsets of {b E B \ £(b) = z + 1} that are linearly independent modulo the 
subspace Si , and let M*+i be a maximal such subset. It is clear that M 
generates the subalgebra B. Moreover, M is a reduced set. 

Hence any subalgebra of A is free in the same variety of algebras (this 
is what is called the Schreier property). In [212], J. Lewin proved that for 
a homogeneous variety of algebras, the Nielsen and Schreier properties are 
equivalent. By using this equivalence, it is easy to see that automorphism 
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groups of the corresponding free algebras (of finite rank) are generated by 
elementary automorphisms (this result was obtained for free Lie algebras 
by P. M. Cohn in [77] and for free nonassociative algebras by J. Lewin 
in [212]). 

Let be a homogeneous variety of linear algebras over a field K. Let 
C be an algebra of the variety 9JI and (y) the free algebra of rank 1 of 
the variety 9ft. We consider the free product B = (y) * C £ Wl. By B\ we 
denote the subspace of elements of B with degree 1 with respect to y. Then 
B\ is a free one-generated C-bimodule in 9ft. Consider C © B\ with the 
multiplication given by 

(a\ + mi)(a 2 + m 2 ) = a\a 2 + a\m 2 + raia 2 , 

where a\,a<i G C, mi, m 2 E B\. We have C © B\ £ 9Jt. For a £ C, by l a 
and r a we denote the universal operators of left and right multiplication, 
respectively, 

b-l a = ab, b • r a = 6a, b £ B\. 

Let {7(C) = U<m(C) be the subalgebra of the algebra End j p(5i) of en- 
domorphisms of the module B\ generated by the set {l,Z a ,r a | a £ C}. 
If the algebra C has the unit element, then we set rq = l\ = 1 in {7(C). 
The algebra U (C) is the universal multiplicative enveloping algebra of C. 
The notion of a C-bimodule in the variety 9ft is equivalent to the notion of 
a right {7(C) -module; see [157]. 

By A = Ffffi(X) we denote the free algebra of the variety 911 with the set 
X = {xi, . . . , x n } of free generators. Let 9Jto be the variety of all algebras 
and T(X) the free groupoid of nonassociative monomials (free Magma) 
without unity element on the alphabet X ; i.e., X C r(X). If u,v £ T(X), 
then u • v £ T(X), where u • v is the formal multiplication on nonassocia- 
tive monomials. We consider the linear space F(X) over K with the basis 
consisting of 1 and the elements of r(7T) with the multiplication given by 

(aa) • (pb) = (a/3) (a • b) 

for all a, /3 £ K , a,b £V(X). The algebra F(X) is the free nonassociative 
algebra (the free algebra of the variety 9 JIq). Let Wo = T(X), A = F(X). 
Then the algebra U(A) is the free associative algebra with the set So = 
{r w P w | w £ Wo} of free generators (for the details, see [140, 388]). 

Let 97li be the variety of all commutative algebras, I the two-sided ideal 
of the free nonassociative algebra F(X) generated by the set {ab — ba | 
a, 6 £ F(X)}. Then the factor algebra A = F(X)/I is the free algebra of 
the variety 9Jli with the set X of free generators (i.e., the free commutative 
nonassociative algebra). 

Suppose that the set T(X) is totally ordered in such a way that, for a, b £ 
r(X), if £(a) > £(b), then a > b (£(u) denotes the degree of a monomial 
u £ r(7T)). We construct the set W\ of all commutative regular monomials 
inductively in the following way. First of all, X C W\. Furthermore, w £ W\ 
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if w = uv, u and v are commutative regular monomials, and u < v. Then 
W\ is a linear basis of A = F(X)/I ([346]). The universal multiplicative 
enveloping algebra U(A) is the free associative algebra with the set S\ = 
{r w | w £ Wi} of free generators [140, 388]. 

Let M 2 be the variety of all anticommutative algebras and J the 
two-sided ideal of the free nonassociative algebra F + (X) without unit 
element generated by the set {aa | a £ F(X)}. Then the factor alge- 
bra A = F+(X)/J is the free algebra of the variety M 2 with the set X 
of free generators (i.e., the free anticommutative nonassociative algebra). 
We construct the set W 2 of all anticommutative regular monomials induc- 
tively: X C W 2 ; w £ W 2 if w = uv, u, and v are anticommutative regular 
monomials, and u < v. Then W 2 is a linear basis of A = F(X)/J ([346]). 
The universal multiplicative enveloping algebra U (A) is the free associative 
algebra with the set S 2 = {r w \ w £ W 2 ] of free generators [140, 388]. 

Let G be an abelian semigroup, K a field of characteristic different from 
two, and e : G x G K* a skew symmetric bilinear form (a commutation 
factor, or a bicharacter); that is, 

£(g,h)e{h,g) = 1, 

e(gi + 92 , h) = e(gi, h) e(g 2 , h), e{g, hi + h 2 ) = e{g , hi) e{g, h 2 ) 
for all g,gi,g 2 ,h,hi,h 2 € G; 

G- = {g G G | e(g,g) = -1}, G+ = {g e G \ e(g,g) = +1}. 

For instance, if G = Z 2 ©Z 2 , K = C, / = (/ 1 , f 2 ),g = ( 01 , g 2 ) e G, then the 
following form is a commutation factor on G : e{f,g) = (— l)(/ 1+ / 2 )(pi+p 2 ). 

= {(0, 0), (1, 1)}, G- = {(0, 1), (1, 0)}. Suppose that n = 2 k , k > 1. Let 
e n be a primitive root of 1, e n = e 2m / n , G = (/) n 0 (g) n the direct sum 
of cyclic groups of order n generated by / and g, respectively, e(f, f) = 1, 
e(f,g) = e n , £{g,g) = —1. Then e is a commutation factor on G. 

M. Scheunert [326] classified all commutation factors on groups Z p 0 
• • • 0Z P , where p is a prime number. A commutation factor e on an abelian 
group G is said to be nondegenerated if {g G G \ £(g, h) = 1 Mh £ G} = {0}. 
A commutation factor e' on a group G' is equivalent to a commutation 
factor £ n on a group G" if there is an isomorphism cp : G' -» G n such that 
£ f (g,h) = £ n {<p{g), (p(h)) for all g, h G G'. A. A. Zolotykh [419] obtained 
a classification (up to an equivalence) of all nondegenerate commutation 
factors on finite abelian groups. 

A G-graded if-algebra R = @ geG R g is a color Lie superalgebra if 

[x,y\ = -e(d(x),d(y))[y,x], [v,[v,v]] =0, 

[x, [y, z\] = [[x, y\,z] + e(d{ x), d(y))[y, [x, z]] 

with d(v) G G_ for G-homogeneous elements x, y,z,v G R, where d(a) = g 
if a G R g . The homogeneous elements of the components R g , g G G+, 
are said to be even, and the homogeneous elements of the components R g , 
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g G G_, are said to be odd. Let R + - 0 9 e G+ Rg, R- - © g€ G_ Rg- 
We consider only G-homogeneous elements, homomorphisms preserving the 
G-graded structure, etc. 

If G = Z 2 and s(f,g) = (— 1)^, then a color Lie superalgebra is a Lie 
superalgebra. If e = 1, then we have a G-graded Lie algebra (if G = {e}, 
then we have a Lie algebra). 

We denote (ad a) (b) = [a, 6] = (a) Ad b for all a, 5 G R. Let char A" = 
p > 2. A color Lie superalgebra R over A" is a color Lie p-super algebra if 
on G-homogeneous components R g , g G G+, we have a mapping x — > x^\ 
d(x [p!) = pd(:r), such that, for all a G if and all G-homogeneous elements 
x,y, z G i? with d(a;) = d(y) G G+, the following conditions are satisfied: 

(ax)^ = a p x^\ (ad(:c^))(z) = [x^,z] = (ad x) p (z), 

(x + y) w = z w +y w + ^2sj(x,y), 

where jsj(x,y) is the coefficient on in the polynomial (ad (tx + 
y)Y-\x). 

If Q is a G-graded associative algebra over if, then [Q\ denotes the color 
Lie superalgebra with the operation [ , ] where [a, b] = ab — e(d(a), d(b))ba 
for G-homogeneous elements a, b G Q- 

If char if = p > 2, and x ^ = x p for all G-homogeneous elements x of Q 
with d(x) G G+, then [Q] with the operation \p\ is a color Lie p-superalgebra 
denoted by [Q] p . 

Let X = {x\ ,...,x n } = U geG^g be a G-graded set (that is, X g fl 
Xf = 0 for g ^ /, d(x) = g for x G X g ), and let K(X) be the free G- 
graded associative if-algebra and L(X) the subalgebra of [if (X)] generated 
by X. Then L(X) is the free color Lie super algebra with the set X of free 
generators. In the case where char if = p > 2, let L P (X) be the subalgebra 
of [if (X)] p generated by X. Then L p ( X) is the free color Lie p-superalgebra 
on X. For more information on free Lie super algebras, we refer to the 
monographs [33, 275]. 

We combine the main properties of free algebras of main types of Schreier 
varieties of algebras in one theorem. 

Theorem 11.1.1 ([77, 195, 212, 243, 244, 245, 345, 346, 369, 403]). 

Let X be a finite set, X = { x \ , . . . , x n }, if a field, char if ^ 2, F = F(X) 
the free algebra without the unity element on the set X of free generators of 
one of the following varieties of algebras over a field if: the variety of all 
algebras, the variety of Lie algebras, varieties of color Lie superalgebras, the 
variety of Lie p-algebras, varieties of color Lie p-superalgebras, or varieties 
of commutative and anticommutative algebras. 

1. Any finite subset of F can be transformed into a reduced subset by 
a finite sequence of elementary transformations ( with cancellation of 
possible zeros). 
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2. Any reduced subset of the algebra F is an independent subset. 

3. The leading part of a polynomial on a reduced subset is a polynomial 
on leading parts of elements of this subset. 

4. Any subalgebra of F is free. 

5. A subset M ofF is independent if and only if it is linearly independent 
modulo the square of the subalgebra of F generated by M (for free 
color Lie p-superalgebras , we add to the square the ideal all p-powers 
of elements of M) . 

6. If \X\ < oo, then the automorphism group of F is generated by 
elementary automorphisms. 

Problem 11.1.2 (P. M. Cohn [77]). Is it true that the automorphism 
group of a free associative algebra of finite rank is generated by elementary 
automorphisms? 

Note that this problem has a positive solution for both polynomial al- 
gebra in two variables (H. W. E. Jung [164], W. van der Kulk [194]) 
and the free associative algebra of rank 2 (A. J. Czerniakiewicz [86], 
L. G. Makar-Limanov [233]). 

Let X = {xi, . . . ,x n }. It follows from Theorem 11.1.1 that free algebras 
F(X) are Hopfian; that is, if the algebra F(X) is generated by elements 
u\, . . . , u n , then these elements form a free generating set of F(X). 



11.2 Schreier Techniques 



O. Schreier [329] introduced very useful techniques to obtain free generators 
for a subgroup of a free group from a special set of coset representatives. 
Later, this technique was extended by many authors for different types of 
free algebras. 

Subalgebras of the free associative algebra are not necessarily free. For 
example, if we consider elements x 2 and x 3 in the free associative algebra 
K(x), then x 2 x 3 = x 3 x 2 . Hence the subalgebra generated by x 2 and x 3 is 
not free. At the same time, we have P. M. Cohn’s result that over a free 
associative algebra submodules of free left modules are free. P. M. Cohn 
obtained this result using a generalization of the Euclidean algorithm in 
the free associative algebra. 

Let K be a field. The family {a* | i £ 1} of elements of the free associative 
algebra K(X) is called left ^-dependent if there exist elements bi e K(X), 
almost all zero, such that 




< ma x{£(ai) + £(bi)} 
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or if some a* is equal to zero. Otherwise the family {a*} is called left 
^-independent. 

An element a G K(X) is said to be left ^-dependent on a family {a*} if 
either a* = 0 or there exist bi G K(X), almost all zero, such that 

l(a-Y2bi a i \ <£(a), £(di) + £(bi) < £(a) 

' i ' 

for all i. Otherwise a is said to be left ^-independent on a family {ai}. 

In the free associative algebra over a field K , the following analog of the 
Euclidean algorithm takes place: 

Theorem 11 . 2.1 (weak algorithm [ 74 , 75 ]). Let a\,...,a n G K(X) 
and£(ai) < • • < £(a n ). Then , if the family {ai, . . . , a n } is left £-dependent, 
there exists i, 1 < i < n, such that the element ai is left dependent on the 
family {a 1 , . . . , a*_i}. 



Note that the right version of this algorithm also takes place in K(X). 

Since the weak algorithm takes place in K(X), the algebra K(X) is a free 
ideal ring (a fir) — i.e., K(X ) is left fir and right fir; that is, left (right) 
ideals of K(X) are free left (right, respectively) A" (A) -modules of unique 
rank. Moreover, in a left fir every submodule of a free left module is free 
[76, 80]. 

Note that a very elegant proof of this fact is given by P. M. Cohn’s 
theorem that the free product of firs is again fir. In [213] J. Lewin, with 
the use of Schreier’s technique, gave a direct proof of the fact that K(X) 
is a fir. 

Using the weak algorithm, P. M. Cohn [77] obtained the following suf- 
ficient condition for a subalgebra of a free algebra to be free. Let B be 
a subalgebra of K{X). We consider K(X ) as a left B-module relative to 
usual multiplication. A system of elements S = (s* | i G 1} of K(X) is said 
to be B-independent if for almost all zero elements a^ 




= ma x{£(vi) +^(s*)}. 



Theorem 11.2.2 ([77]). If B is any subalgebra of K(X) such that K(X) 
is free as a left B -module, with a B -independent basis, then B is a free 
associative algebra over K. In particular, if B is a subalgebra of K(X) 
such that K(X) satisfies the weak algorithm as a B -module, then B is 
a free subalgebra of K(X). 



For more information on free ideal rings, we refer to P. M. Cohn’s 
monograph [80]. 

Suppose that F is a free group of rank n, and let H C F be a subgroup of 
a finite index. Then H is also the free group; moreover, the rank m of H is 
determined by Schreier’s well-known formula [329]: m — 1 = (n— 1) • |F : H |. 
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J. Lewin [213] obtained the following analog of Schreier’s formula for free 
associative algebras. For a left ideal M of K{X), we denote by rank(M) 
the rank of the free left K (X) -module M. 

Theorem 11.2.3 ([213]). Let \X\ = n and let I be a left ideal of K(X) 
such that dim k(K(X)/I) = k < oo. Then I is a free left K(X) -module, 
rank(7) = k(n — 1) + 1. 

For free Lie p-algebras, G. P. Kukin [192] obtained the following analog 
of Schreier’s formula. 

Theorem 11.2.4 ([192]). Let K be a field, p = char if > 2, \X\ = 
N < oo. Suppose that H is a subalgebra of the free Lie p-algebra L P (X), 
dim L p (X)/H = t< oo. Then rank (if) = p t (N - 1) + 1. 

A. A. Mikhalev [243, 244, 245] obtained Schreier-type formulas for free 
Lie superalgebras and p-superalgebras. 

Theorem 11.2.5 ([243]). Let K be a field, char if ^2,3 , L = L(X) = 
L+ ® L- the free color Lie superalgebra, rank (L) = \X | = N < oo, and 
let H be a G -homogeneous subalgebra of L, H = H+ ® i7_, i7 + = L+, 
dim {L-/HJ) = 8 < oo. Then rank(i7) = 2 S (N - 1) + 1. 

Theorem 11.2.6 ([245]). Let if be a field, p = char if > 3, \X\ = 
N < oo. Suppose that H = ii+ ® i7_ is a G -homogeneous subalgebra 
of the free color Lie p-superalgebra L P (X), dim (Iq_/ii + ) = t < oo, and 
dim (L p _/H_) = s< oo. Then rank (H) = 2 s p t (N - 1) + 1. 

A. A. Mikhalev showed that any subalgebra of finite rank of L(X) is 
completely defined by its (nontrivial) even component. 

Theorem 11.2.7 ([33, 275]). Let if be afield, char if ^ 2,3, |X_| > 1, 
and let A , B be G -homogeneous subalgebras of finite rank in the free color 
Lie superalgebra L(X), A = A+ ® B = B+ ® B_. If A- = ^ {0}, 

then A — B. 

In the study of subalgebras of free algebras, the elimination of variables 
plays a very important role. 

Let x G X, v G X+, and z E X_, where X+ = U g eG + ^ 9 ’ = 

|J ge c- We denote 

Z(x) = {y,y( Adx)" \yeX\ {x}, n E N} C L(X); 

ZP(v) = {y,v p ,y(Adv) n \ yeX\ {x}, n = 1, . . . ,p - 1} C L P (X); 
W(z) = {y,[y,z],[z,z\ | yeX\{x}} C L(X). 

For free Lie algebras, the elimination theorem was obtained by M. Lazard 
[202]. 
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Theorem 11.2.8 ([202]). Let L{X) be the free Lie algebra , x G X. Then 
Z(x) is an independent subset of L(X) and 

L{X) = Kx®L(Z(x)). 

W. Magnus [230] obtained elimination theorems for free associative rings. 
Elimination of variables in free color Lie superalgebras and p-superalgebras 
was considered by A. A. Mikhalev. 

Theorem 11.2.9 ([243, 244, 245, 33, 275]). 

1. Let L(X) be the free color Lie superalgebra , x G X, z G X_. Then 
Z = Z(x) and W = W(z) are independent subsets of L(X). The ideal 
of L(X) generated by {y G X | y ^ x} is equal to L(Z(x)). Moreover, 
L(X) = L(x) 0 L(Z(x)) is a semidirect product of the free algebras 
L{x) and L(Z(x)) with the adjoint action of L(x) on L(Z(x)). We 
also have L(X) = Kz 0 L(W(z)) as vector spaces. 

2. Let L P {X) be the free color Lie superalgebra, x G X, v G X+, z G 
X-. Then Z(x), ZP(v), andW(z) are independent subsets ofL p (X). 
Moreover, L P (X) = L p (x) © L p (Z(x)) is a semidirect product with 
the adjoint action and L P (X) = Kv 0 L p (ZP(v)), L P (X) = Kz 0 
L p (W(z)) as vector spaces. 

The elimination theorem for free Lie superalgebras was obtained also by 
J. Desarmenien, G. Duchamp, D. Krob, and G. Melangon [91]. Recently 
N. Dobrynin [98, 99] obtained some generalizations of the elimination theo- 
rem for free Lie super algebras. For free partially commutative Lie algebras 
elimination theorems were obtained by G. Duchamp and D. Krob [114], and 
for free partially commutative superalgebras — by N. Dobrynin [98, 99]. 

Note that a straightforward analog of Schreier’s formula for the free Lie 
algebras does not exist. It is well-known that the rank of a subalgebra of 
finite codimension of a free Lie algebra is infinite; see, for example, [32]. But 
recently V. M. Petrogradsky found a form of Schreier’s type formula for free 
Lie algebras and super algebras in terms of formal power series [303, 305, 99]. 
He also established versions of the Schreier formula for these algebras using 
exponential generating functions (complexity functions) [302] . Recently he 
obtained Schreier formulas for free nonassociative algebras and free com- 
mutative and anticommutative nonassociative algebras [306]. We illustrate 
here his idea for free Lie algebras. 

Let X be at most a countable set with a weight function wt : X -» N 
(where N is the set of positive integers) such that 

oo 

X = |^J Xi , Xi = {x(EX\wtx = i}; \X{\ < oo, i G N. 

t=i 

We call such a set finitely graded. For any monomial y = x^ . . . Xi n , x^ G 
X , we set wt y = wt H (- wt Xi n . Suppose that Y is a set of monomials 
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in X. Then we denote Y{ = {y G Y | wt y = i}, i £ N. Also, we define the 
Hilbert-Poincare series of Y as Hx(Y,t) = YliLi I Yi\t l . 

Let A = A(X) be an algebra generated by a finitely graded set X, and 
suppose that A inherits the gradation from X. Let V C A be a homogeneous 
subspace; that is, V = Vi with V% = {v G V | wt v = i}. We define the 
Hilbert-Poincare series of V as 'HxiV.t) = YhL i dim (Vi)f. We sometimes 
omit t, X and write %(y), 

Now let Y C A be a not necessarily homogeneous subset. For v E A, 

consider its decomposition into homogeneous components v = V\ H \-v n , 

wt(vi) = i, i = 1, . . . , n, v n ^ 0. In this case, we write deg(^) = n, gr(^) = 
v n . We define 

oo 

n x (Y,t) = ^2 \ Y n\t n , where Y n — {yeY\ deg (y) = n}. 

n=l 

Given a subspace V C A, one has a filtration 0 = V° C V 1 C . . . with 
V i = {v e V | deg(v) < i}, i = 0,1,2,.... Let gr(V) = ®^° =1 gr n (V), 
where 



gr n (V) = (gr(^) | v 6 V, deg(^) = n}U0 = V n /V n 1 

denote the associated graded space. In the nonhomogeneous case, the 
Hilbert-Poincare series is defined as follows: 

oo 

H X (V, t) = H( gr V,t) dim K (gr n (V))t n -, 

n = 1 
oo 

H X (A/V,t) = H(A/giV,t ) = (i4„gr n (V))i n . 

n=l 

We consider all series as elements of the ring of the formal power series 

Z[[i]] = {f;M n |&„ez], 

^ n=0 ' 

By Zofftll we denote a set of series with bn = 0. We consider the operator 
£:Z 0 [[t]]^Z[[t]} 



OO OO OO 1 

S : ¥>(t) = 52 bntn = 52 = II d _ t n)b n • 

n=l n=0 n=l ' ' 

Let L be a Lie algebra generated by X, let U (L) be its universal enveloping 
algebra, and suppose that 

oo oo 

H x (L,t) = Y,bnt n , n x (U(L),t) = '^ant n . 

71=1 71=0 

It is well-known that / Hx{U{L)) = £(Hx(L)) [384]. 
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Theorem 11.2.10 ([303]). Let L be a free Lie algebra freely generated by 
a finitely graded set X. Suppose that H is a subalgebra and Y is the set of 
free generators for H. Then 

H(Y) - 1 = (H(X) - 1) • £(H(L/H)). 

For more information on Schreier techniques, we refer to the following 
articles and monographs: [32, 33, 34, 57, 91, 99, 189, 190, 192, 195, 198, 
202, 212, 213, 227, 230, 231, 243, 244, 245, 248, 252, 275, 302, 303, 304, 
305, 306, 313, 322, 329, 345, 346, 369, 388, 392, 403, 414]. 



11.3 Free Differential Calculus 



R. H. Fox [134] introduced free differential calculus in free group rings; see 
Section 1.4. Let F = F(X) be the free group on a set X, Z F the integral 
group ring of F, and A^ the augmentation ideal of Z F (i.e., the kernel of 
the natural homomorphism 



a: Z F 






Hi 



where n* G Z, fi G F). The Fox partial derivation with respect to Xi is 
a mapping di : Z F — » Z F that satisfies the following conditions: 



d{ {xj ) — Sij , 

di(uv) = udi(v) + a{v) di(u), 
di(ku + Iv) = kdi(u) + ldi(i >), 



u,v e ZF, fc,Z g Z. 

There is another interpretation of di as follows. The ideal is a free left 
ZF-module with a free basis {(xi — 1) | 1 < i < n}, and the mappings di 
are projections to the corresponding free cyclic direct summands. Every 
element u G Ap can be uniquely written in the form 

n 

U = ^ di(u)(xi - 1 ). 
i= 1 



11.3.1 Free differential calculus and Schreier varieties 

As before, by A = F$n(X) we denote the free algebra of the variety 971 with 
the set X = {#i, . . . , x n } of free generators. 

Let I a be a free right U (A)-module with a basis yi, . . . , y n , 

I A = yiU{A)®---®y n U(A). 

The linear mapping V: A I A given by V(xi) = yi, 1 < i < n, 



V(ab) = V{a) • r b + V(b) • Z a , 
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a, b G A, is called the universal derivation of the algebra A in the variety 9Jt. 
The partial derivatives of an element / of A are uniquely determined 
by 



We set 



©(/) = £ 

i = 1 




9(f) = 



df_ df_y 

dxi ’ ” ’ ’ dx n ) 



where T is the operator of transposition. From the definition, we get 
-£^(xj) = 5ij , the Kronecker symbol, 

d x du dv 

— (uv) = — r v + — 1 U1 u,veA. (11.1) 

OXi OXi OXi 

For a subalgebra H of the algebra A by J#, we denote the {7(A)- 
submodule of the module I a generated by the elements D(h ), h E H. 
A variety 9Jt has the property of differential separability for subalgebras if 
for any subalgebra H of A and a € A, D(a) 6 Jh if and only if a e H. 



Theorem 11.3.1. The following varieties have the property of differential 
separability for subalgebras: 



1. the variety of all algebras; 

2. the variety of all commutative algebras (char AT ^ 2); 

3. the variety of all anticommutative algebras ; 

4. the varieties of Lie (super) algebras and p-(super) algebras; 

5. the variety of algebras with zero multiplication. 

The first time the property of differential separability for subalgebras for 
Lie algebras was mentioned was in [385] (for Lie super algebras, see [249]). 
In general form, this theorem was proved by U. U. Umirbaev in [388]. 

For Lie algebras and superalgebras, the property of differential separa- 
bility for subalgebras follows from the Poincare-Birkhoff-Witt theorems. 

Let L be a color Lie superalgebra, U a G-graded associative AT-algebra 
with 1, and 5: L — > [U] a homomorphism of color Lie superalgebras. We 
say that the algebra U (with 5) is the universal enveloping algebra of L (we 
denote it U(L )) if for any homomorphism a of the color Lie superalgebra L 
into a color Lie superalgebra [R] (with the same e and G) for some G- 
graded associative algebra R with 1 there exists a unique homomorphism 
0 : U R of G-graded associative algebras with 1 such that a = 05. Let K 
be a field, p = char K > 2, L a color Lie p-superalgebra, and U a G-graded 
associative K - algebra with 1, S : L — > [U] p a homomorphism of color Lie p- 
super algebras. We say that U (with 5) is the restricted universal enveloping 
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algebra of L (notation u(L)) if for any homomorphism a: L -* [R] p of 
color Lie p-super algebras, where R is a G-graded associative if-algebra 
with 1 (with the same e and G), there exists a unique homomorphism 
0: U -> R of G-graded associative if-algebras with 1 such that a = 05. 
Suppose now that L is a color Lie superalgebra with a G-homogeneous 
basis X = X+ U X_, K (X) is the free G-graded associative algebra, and 
I is the two-sided ideal in K(X) generated by the elements of the form 
ab — e(d(a),d(b))ba — [a, b\ for all homogeneous a, b G L. Consider the 
canonical mapping 5: L — > K(X)/I = U(L). It is clear that U(L) is the 
universal enveloping algebra of L. Let < be a total ordering of X. 

Theorem 11.3.2 (PBW type). The universal enveloping algebra U(L) 
constructed above has a G-homogeneous basis consisting of 1 and all mono- 
mials 5(xi) . . . 5(x n ) where n G N, Xi G X, xi < Xi+\, X{ ^ Xi+ \ with 
Xi G for all i = 1, . . . , n — 1 (in particular, 5 is an embedding). 

Let K be a field, char K = p > 2, L a color Lie p-superalgebra with a G- 
homogeneous linear basis X , J the two-sided ideal of K(X) generated by 
elements ab— s(d(a)a, d(b))ba—[a , b] (where a, b are homogeneous, a,b e L), 
and a p — a& 1 (a G L g , g G G+). Let 5: L — K(X)/J be the canonical map- 
ping. It is clear that K(X)/J = u(L ) is the restricted universal enveloping 
algebra for L. 

Theorem 11.3.3 (PBW type). The monomials 5(xi) Xl . . . 5(x n ) xn , 
where Xi G X, x\ < • • • < x n , 0 < A* < p — 1 for X{ G X+, A* = 0, 1 
for Xi G X- give us a linear basis of u(L). 

It is clear that the universal enveloping algebra of the free color Lie 
superalgebra L(X) is the free associative algebra K(X). This algebra is also 
the universal multiplicative enveloping algebra of L(X). For the restricted 
universal enveloping algebra of the free color Lie p-superalgebra L P (X ), we 
have u(L p (X)) = K(X). This algebra is also the universal multiplicative 
enveloping algebra of L P (X). For more results concerning PBW theorems 
for Lie super algebras, we refer to the monographs [33, 275, 327]. 

Theorem 11.3.4. Let K be a field, L a color Lie superalgebra, and B 
a subalgebra of L. Then B is differentially separable; i.e., if J is the left 
ideal of U (L) generated by B, then LC\ J = B. 

In the case where char K = p > 2, let M be a color Lie p-superalgebra 
and D a subalgebra of M. Then D is differentially separable ( relative to 
u(M)). 

Proof. Let E = {e i, e 2 , • • • } be a G-homogeneous ordered K - linear basis 
of the subalgebra B. We extend the subset £ up to a G-homogeneous 
ordered X-linear basis E U H of L such that H = {hi, / 12 , . . . } and hi < e, 
for all i,j. By Theorem 11.3.2, the set of all monomials 
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where 



foil — * ’ * — him J ^il — * * * — C in 5 

i r ^ i r+ i with d(fe» r ) e G_, ^ j s +i with d(e js ) e G_, gives us a if- 

linear basis of K(X). Since B is a subalgebra of L, these monomials with 
n > 1 form a if -linear basis of J. Therefore Ln J = B. 

The same arguments work for the color Lie p-superalgebra M (we need 
only use Theorem 11.3.3). ■ 

Using the property of differential separability of subalgebras, it is possible 
to prove that free Lie algebras and superalgebras are residually finite. 

Theorem 11.3.5 ([249, 385]). Let L(X) be a free color Lie superalgebra 
over afield K (in the case char if = p > 2, we consider also a free color Lie 
p- superalgebra L P (X)). Then the algebra L = L(X) (L = L P (X), respec- 
tively) is residually finite; i.e., if a is a G-homogeneous element of L, B is 
a G-homogeneous finitely generated subalgebra of L, a £ B, then there ex- 
ists a finite- dimensional color Lie (p-) superalgebra H and an epimorphism 
(p: L H such that <p(a) £ <p(B). 

Proof. It is sufficient to prove the statement in the case where \X\ < oo. At 
the beginning, we consider the free color Lie superalgebra L = L(X). The 
free G-graded associative algebra K(X) is the universal enveloping algebra 
of L. Consider the left ideal J of K(X) generated by the subalgebra B. 
Since B is a finitely generated subalgebra, J is a finitely generated left 
ideal of K(X). Since a e L and a ^ B, by Theorem 11.3.4 a £ J. Since the 
ideal J is finitely generated, there exists a left G-graded ideal J' of finite 
codimension in A(X) such that a ^ J'. 

Consider the finite-dimensional left U(L )~ module M = U(L)/ J', where 
U(L) = K(X). Let Ann (M) be the annihilator of the module M and 
R = U(L)/ Ann(M). Then R is a finite-dimensional G-graded associative 
algebra, M is a faithful jR-module, and Ann (M) C J' since 1 e U(L). Let 
£: U(L) -» R be the canonical epimorphism. It is obvious that H = [£(L)] 
is a finite-dimensional color Lie superalgebra. Assuming now that £(a) € 
£(£?), one obtains for (ui, . . . , u r } a set of generators of the subalgebra B 
such that £(a) belongs to the left ideal of R generated by £(ui), . . . ,£(u r ). 
Therefore there exist elements v \ , . . . , v r G U (L) such that 

r 

£(<*) = 

2 — 1 

in other words 

r 

a e Ann (M) C J', 

2=1 

and thus a E J'. This contradiction completes the proof of our theorem in 
the case L = L(X). 
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In the case char If = p > 2 and L = L P (X), the proof is the same using 
the fact that the free G-graded associative algebra K(X) is the restricted 
universal enveloping algebra of L. ■ 

In group theory, a well-known theorem by M. Hall shows that free groups 
are residually finite. G. V. Kryazhovskih [188] showed that free nonasso- 
ciative algebras and free commutative and anticommutative nonassociative 
algebras are residually finite. This result for free Lie algebras over fields of 
positive characteristic and for free restricted Lie algebras was obtained by 
G. P. Kukin [193]. A similar statement does not hold for free rings. In the 
review article [126] is the following example, which is due to K. Mandel- 
berg. If F is a free ring on one generator x, then in any homomorphism 
of F onto a finite ring, the subring B generated by 2x and 2x 2 + x maps 
onto the image of F. At the same time x £ B. 

The following result by U. U. Umirbaev gives a necessary condition for 
a variety of algebras to be Schreier. 

Theorem 11.3.6 ([388]). If Tl is a homogeneous Schreier variety of 
algebras , then 

1. for any free algebra A of DJI the universal multiplicative enveloping 
algebra U (A) is a free associative algebra ; 

2. for any homogeneous subalgebra H of a free algebra A of the variety 971 
the algebra U(A) is a free right U (H) -module. 

If char K = 0, then a variety satisfying conditions 1 and 2 is Schreier . 

Point 1 of Theorem 11.3.6 for free nonassociative algebras and for free 
commutative and anticommutative nonassociative algebras was proved by 
A. T. Gainov [140]. 

A method to test that a variety of algebras is Schreier is given by the 
following theorem. 

Theorem 11.3.7 ([388]). Let m be a homogeneous variety of algebras 
such that: 

1. For any free algebra A of 97t, the universal multiplicative enveloping 
algebra U (A) is a free associative algebra. 

2. The variety 971 has the property of differential separability for 
subalgebras. 

Then 97 1 is a Schreier variety. 

It follows that: 

Theorem 11.3.8 ([388]). The variety of algebras defined by the identity 
x • x 2 = 0 is Schreier. 

Problem 11.3.9. To give the complete classification of Schreier varieties 
of algebras. 
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An algebra A over a field K with an anticommutative bilinear operation 
[x, y] and a trilinear operation A(x, y , z) is called an Akivis algebra if 



[[x, y\,z\ + [[y, z],x] + [[z, x],y]= A{x, y, z) + A(y, z, x) + A(z, x, y) 

- A{y, x, z) - A(x, z, y) - A(z, y, x). 



These algebras were introduced by M. A. Akivis as tangent algebras of 
local analytic groups. I. P. Shestakov and U. U. Umirbaev [338] proved 
that the variety of Akivis algebras is Schreier, automorphisms of finitely 
generated free Akivis algebras are tame, the occurrence problem for free 
Akivis algebras is decidable, finitely generated subalgebras of free Akivis 
algebras are residually finite, and the word problem is decidable for the 
variety of Akivis algebras. 

For any element a of the free associative algebra K(X), we have the 
unique presentation in the form a = a • 1 + x\a\ + X2&2 + . . . , where only 
a finite number of elements a* G K(X) are nonzero, a G K. We call the 
element a t the right Fox partial derivative of the element a by and 
we use the notation a* = Thus we have the operators 

These operators are exactly linear mappings K(X) -» K(X) such 
that -£^(xj) = 5ij and 



d , N d / N , N d , 



where a: K(X) — >• K(X) is the homomorphism defined by a (xi) = 0 for all 
Xi G X. For one of the applications of these derivatives, see Section 9.2.1. 

For all a G L(X) and b G K(J T), we set l a -b = — e(a, b)ba and a-r^ = a*6. 
Then K(X) is an L(X)-bimodule. It is clear that the universal multiplica- 
tive enveloping algebra of L(X) is exactly the free G-graded associative 
algebra K(X) with the actions l a and r a defined above (at the same time, 
K(X) is the universal enveloping algebra of the color Lie super algebra 
L(X)). The derivation 



n 

D:L(X)^I A = @d Xi K(X) 

i = 1 



given by 



D(a) = Y ' dxi- ~ 

h dxi 



for all a e L(X) (only a finite number of summands are nonzero) is the 
universal derivation of L(X). 

For more information on Schreier varieties of algebras, we refer to the 
articles [388, 392] (for the varieties of linear 11-algebras, see [26, 38]). 

Free differential calculus can be applied to a study of free abelian ex- 
tensions in a variety 21. Let A be an algebra in 9 3 with a fixed set of 
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generators X. We say that an algebra B in 53 is a free abelian extension 
of A in 53 if 

1. B is generated by the same set X. 

2. There exists a surjective homomorphism 7 : B ^ A that is identical 
on X. 

3. If C is an algebra in 53 generated by the set X having a surjective 
homomorphism rj: C — > A that is identical on X, then there exists 
a homomorphism £ : B C such that rj£ — 7 . 

Take a free 53-algebra A with the set X of free generators. Since A is 
generated by the set X, there exists a surjective homomorphism of algebras 
p: F —> A that is identical on X. Denote by I the kernel kerp. It is not 
difficult to show that the free abelian extension B is precisely F/I 2 , where 
1 2 is the ideal in F generated by all products ab , a, b £ I. Thus, free abelian 
extensions are analogs of wreath products in group theory. An extension 
of a study of free abelian extensions to any congruence-modular variety in 
universal algebra and a development of generalized free differential calculus 
is exposed in [23]. 



11.3.2 Ranks of subalgebras 

Using free differential calculus (universal derivations), A. A. Mikhalev, 
V. Shpilrain, and A. A. Zolotykh [257] obtained matrix algorithms to 
find ranks of subalgebras of free Lie algebras. A. A. Mikhalev and 
A. A. Zolotykh [279] constructed such algorithms for subalgebras of free Lie 
superalgebras and free Lie p-super algebras. Recently, K. Champagnier [67] 
obtained similar results for ranks of subalgebras of free nonassociative 
algebras and free (anti) commutative nonassociative algebras. 

Let K be a field, char if ^ 2, and L(X) the free color Lie superalgebra 
(if char if = p > 2, also let L P {X) be the free color Lie p-superalgebra). 
For the algebra A = L(X) (A = L P (X )), the universal multiplicative 
enveloping algebra 17(A) of A is the free associative algebra K(X). Let 
H = {/ii, . . . , h m } be a subset of G-homogeneous elements of A. Consider 
the matrix J(H ) = (^-), 1 < i < n, 1 < j < m. Here partial deriva- 
tives ^ are components of the (right) universal derivation. Consider the 
columns d(hj ), 1 < j < m, of the matrix J{H) = (<9(/ii), . . . , d(h m )). 

Theorem 11.3.10 ([279]). A subset H = {/ii, . . . , h m } of A is indepen- 
dent if and only if the columns of the matrix J(H) are right independent 
over U(A). 

Theorem 11.3.11 ([279]). Let H = {hi , . . . , h m } C A. Then the rank 
of the subalgebra of A generated by H is equal to the rank of the free right 
U (A) -submodule ofU(A) n generated by the columns of the matrix J(H). 
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Theorem 11.3.10 gives an effective algorithm to decide whether a given 
subset of A is independent or not. Note that U. U. Umirbaev [386] 
showed that this problem for free associative algebras is algorithmically 
undecidable. Theorem 11.3.11 gives an effective algorithm to find the 
rank of a finitely generated subalgebra of A. These algorithms are based 
on the right weak algorithm. Another algorithm could be derived from 
Theorem 11.1.1. Indeed, using combinatorics on words, by elementary 
transformation we could transform the set H to a reduced set. The 
cardinality of this set is the rank of the subalgebra of A generated by H. 



11.4 Stable Equivalence 

An element u of a free algebra F is said to be primitive if it is an element 
of some set of free generators of the algebra F. A set of nonzero pairwise 
distinct elements of F is said to be a primitive system of elements if it is 
a subset of some set of free generators of F. The rank of u 6 F is the small- 
est number of generators from X on which an element p(u) depends, where 
ip runs through the automorphism group of F (in other words, rank(u) is 
the smallest rank of a free factor of F containing u). In the same way, the 
rank of a system of elements is defined. 

In this section, we are interested in the following problems. 

Problem 11.4.1. Let A be a free subalgebra of a free algebra F. Given 
two elements u,v G A such that there exists an automorphism (p of F with 
< p(u ) =v, does there exist an automorphism 'ip of A such that tf(u) = v7 

Problem 11.4.2. Let A be a free subalgebra of a free algebra F,ua prim- 
itive element of F, and u E A. Does it imply that u is actually a primitive 
element of the free algebra A? 

For free groups F of finite rank, the affirmative answer to Problem 11.4.1 
with A a free factor of F and to Problem 11.4.2 follows from [227, 
Propositions 1.4.17 and 1.5.4]. V. Shpilrain and J.-T. Yu [365] solved Prob- 
lem 11.4.2 for polynomial algebras in two and three variables with A being 
a free factor of F\ see Proposition 6.0.4. 

Problem 11.4.2 has a positive solution for the main types of Schreier 
varieties of algebras. 

Let K be a field, char K ^ 2, X = {xi, . . . , # n }, and F = F(X) the free 
algebra without the unity element on the set X of free generators of one of 
the following varieties of algebras over a field K : the variety of all algebras, 
the variety of Lie algebras, varieties of color Lie super algebras, the variety 
of Lie p-algebras, varieties of color Lie p-superalgebras, and varieties of 
commutative and anticommutative algebras. 
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Proposition 11.4.3 ([265]). Let A = {ai, . . . , a/} be a primitive system of 
elements of F and B a finitely generated subalgebra of F such that A C B. 
Then A is a primitive system in B. 

Proof. Let H be a finite generating set of B, and H f = H U A. Using 
a finite number of elementary transformations of B 7 , we will come to a free 
generating set of B. We may suppose that A C X. We may use only the 
following elementary transformations: if hi , . . . , h s E H' and the leading 
part h\ of the element h\ belongs to the subalgebra generated by the lead- 
ing parts 7^2 , • • • , h ° s , h\ = 0 (^ 2 ? • • • K)i ^ en we set = hi — 0 (/i 2 , • • • h s ). 
It is clear that using these elementary transformations (with possible can- 
cellation of zeros) we may come to a free generating set Y of B such that 
A CY. Thus A is a primitive system of elements of B. ■ 

Now we consider Problem 11.4.1. At first we show that in general form 
this problem has a negative solution. We illustrate this for free Lie alge- 
bras. Let B = L(X) be the free Lie algebra, where X = {x,y, z}, A the 
subalgebra of B generated by the elements a = x + [y, z], b — y, c— [x, y\. 
The set (a, b, c} is reduced (relative to the ordinary degree function). By 
Theorem 11.1.1, this set is an independent subset of A. Let p be the au- 
tomorphism of B given by p(x) = a, p(y) = y , <p(z) = z. Let u = c and 
v = [a, b]. Then p(u) = v . At the same time u, v E A, u is a primitive ele- 
ment of A. For any automorphism xjj of A, the element ^(u) is a primitive 
element of A. Thus i/)(u) ^ v — p(u) = [a, b] E [A, A]. We solve Prob- 
lem 11.4.1 in the affirmative supposing that A is a free factor of the free 
algebra F(X). 

Theorem 11.4.4 ([267]). Let X and Y be nonempty sets , X fl Y = 0, 
Ui,Vi E F(X), Ui 7 ^ 0 , Vi ^ 0 , i = 1 ,...,m. Suppose that there is an 
automorphism p of the algebra F(XUY) such that <p(ui) = Vi, i = 1, . . . , m. 
Then there is an automorphism ^ of F(X) with i/j( U { ) = Vi, i = 1, . . . ,m. 
In other words, if two finite systems of elements of the free algebra F(X) 
are stably equivalent, then they are equivalent. 

Proof. Let X = {x ±, . . . , x n }, U = {u \, . . . , u m }, and rank f(x)(U) = L 
We may suppose that elements of U depend only on x \ , . . . ,x/ E X. On 
the contrary, suppose that the system of elements $ = {p{x i), . . . , <p(xi)} 
depends on a free generator y E Y\ Let d = max^ = i v<<5m deg(i^). Consider 
the generalized degree function /^onlUF given by p,{x) = 1 with x E X, 
p(z) = 1 with z€Y,z^y, and /i(y) = d + 1. 

Using a sequence of elementary transformations, we transform the set $ 
to a /i-reduced set $' = (<//(xi), . . . , <p'(xi)}. Since p is an automorphism, 

is a primitive system of elements of F(XUY). It is clear that $' depends 
on y. We may assume that the element p f (xi) depends on y. The sequence 
of elementary transformations induces the sequence of elementary 

automorphisms 9 of F(x i, . . . , xi), namely p f (xi) = p{9{xi)), i = 1, . . . , /. 
Since rank f(X)(U) = l and Vi = <p(ui), i = 1, . . . ,m, at least one element 
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of V = {^i, . . . , Vm} depends on <p f (xi). One may assume that v\ depends 
on p'(x i). Since Vi G F(X), ja(v i) = deg(ui). At the same time, is 
a /i-reduced set and n(<p'(x i)) > p,{y) = d+ 1. By Theorem 11.1.1, 

deg(vi) = fi(v i) > fj,((p'(xi)) > d + 1 > d = max {deg(u*)}, 

and we have come to a contradiction. This contradiction shows that actually 
$ C F(X). 

Since <p is an automorphism, $ is a primitive system of F(XUY). Since 
4> C F(X), by Proposition 11.4.3 we get that $ is a primitive system of 
F(X). Let {xj+i, . . . ,x n } be a complement of with respect to a free 
generating set of F(X). Now we consider the automorphism ^ of F(X) 
given by ^(xi) = <p(xi ), i — 1, . . . , Z, and f i/j( Xj ) = Xj, j = l + 1, . . . , n. It 
is clear that ^ is an automorphism of F(X) and ^(ui) = i = 1, . . . , m. 



Theorem 11.4.5 ([267]). Let X and Y be nonempty sets , X C\Y = 0, 
and U a finite set of nonzero elements of F(X). Then 

rank F(x) (*7) = rank F(Xur) (l7). 

In other words , the rank of the system U is equal to its stable rank. 

Proof. Using free differential calculus, matrix criteria for systems of el- 
ements of F(X) to have a given rank are obtained in Section 12.4 of 
Chapter 12, Theorems 12.4.7-12.4.10 (free Lie algebras and superalgebras 
were considered over fields of characteristic zero). The statement of the 
theorem follows from these criteria. Here we give the proof without use of 
free differential calculus (and without restriction of the characteristic of the 
ground field for free Lie algebras and superalgebras). 

Let l = ranki?(x)(U) and k = rankp^xu y)(U). It is clear that k < l. Sup- 
pose that k < 1. Then there is an automorphism ip of F(XUY) such that the 
set V = p(U) depends only on free generators xi, . . . , Xk G X. In particular, 
V C F(X). By Theorem 11.4.4, there is an automorphism ^ of the algebra 
F(X) such that V = ^(U). Hence rankj?(x)(U) = mnk F (x)(U) = l > fc, 
and we have come to a contradiction (the set V depends only on xi ,...,£&). 
It completes the proof. ■ 



11.5 The Rank of an Endomorphism 

In this section, we consider the following problem. 

Problem 11.5.1. Let (p be an injective endomorphism of a free algebra F 
and 0 ^ u G F. Is it true that rank(<£>(u)) > rank(u)? 

In order to solve this problem, we define the rank of an endomorphism 
of a free algebra of Schreier variety of algebras and prove in Theorems 
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11.5.3 and 11.5.4 that an injective endomorphism of maximal rank does 
not change the rank of systems of elements with maximal rank. 



Lemma 11.5.2. Let U be a system of elements of a free algebra F(X). Sup- 
pose that a system V is obtained from U by a finite sequence of elementary 
transformations. Then rank(V) = rank(f/). 

Proof. Suppose that rank(F) < rank(C7). Then there is an automor- 
phism p of F(X) such that the system of elements (p(V) depends only 
on rank(F) free generators from X. Since elementary transformations are 
invertible, the set U belongs to the subalgebra of F(X) generated by V. 
Therefore the set p(U) belongs to the subalgebra of F(X) generated by 
the set p(V), and 

rank({7) = rank(</?({7)) < ra,nk(p(V)) = rank(F) < rank(/7). 

This contradiction shows that rank(F) > rank({7). Since there is a sequence 
of elementary transformations V — » J7, we get rank(J7) > rank(F). Thus 
rank(F) = rank(C/). ■ 



Let ip be an endomorphism of F(X ), where X = {xi, . . . , x n }. For 
the rank of ip (rank(</?)), we take the rank of the system of elements 
p(X) = {p(xi), ... , p(x n )}. This notion does not depend on the choice of 
free generators of the free algebra F(X). Indeed, let Y = { 2/1 , - • - , yn} be an- 
other set of free generators of F(X). By Theorem 11.1.1, there is a sequence 
of elementary automorphisms 0: X — > Y. This sequence induces the se- 
quence of elementary transformations 0' of the set p(X), 0' : Y = p(X) — > 
<f(Y) = Mj/i)> ■ ■ • ,<p(i/n)}- By Lemma 11.5.2, rank(y>(F)) = rank(<^(X)). 

Let p be an endomorphism of F(X), 0 ^ u G F(X). If u or p is not 
of maximal rank, or p is not injective, then in general there is no relation 
between rank(<^(u)) and rank(u). We illustrate this for free Lie algebras. 

Let L(X) be the free Lie algebra, where X = {x, y}, p the endomorphism 
of L(X) given by p(x) = x, p(y) = 0, and u = x + [x,y\. Then p(u) = x 
and rank (p(u)) = 1 < 2 = rank(u) = \X\. If 0 is the endomorphism of 
L(X) given by 0(x) = u, 0(y) = 0, then 0(u) = u, and rank(0(u)) = 2 = 
rank(u). If v = x, then 0(v) = u, and rank(0(u)) = 2 > rank(^) = 1. 

If p is an injective endomorphism of maximal rank, but rank(u) < 
n = |X|, then we consider the following examples. Let X = {x, y, z,w}, 
and let p be the endomorphism of the free Lie algebra L(X) given by 
tp( x ) = X, <p(y) = [x,y\, <p(z) = [[x,y],y], <p(w) = [z,w\. Since the 
set {(p(x),(p(y),(p(z),ip{w)} is reduced relative to the ordinary degree, 
ip is injective. Let ui — [[x, y], z], u 2 = [x,y], and u 3 = [x,w\. Then 
<p(ui) = [[x,[x,y]],[[x,y],y]], <p(u 2 ) = [x,[x,y]}, <p(u 3 ) = {x,[z,w}}, and 
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we get 

rank(</?(ui)) = 2 < 3 = rank(ui), 
rank((p(u 2 )) = 2 = 2 = rank (^ 2 ), 
rank(</?(ii 3 )) = 3 > 2 = rank(w 3 ). 

In the case where p is an injective endomorphism of F(X), 0 ^ u E 
F(X), both u and <p have maximal rank and we are able to prove that 
rank(</?(ix)) = rank(u). 

Theorem 11.5.3 ([267]). Let X = x n }, 0 t ^ u E F(X), and 

rank(u) = n. Let also (p be an injective endomorphism of F(X), rank(^) = 
n. Then rank(</?(u)) = rank(u) = n. 

Proof. Suppose that rank(</?(u)) < n. Then there is an automorphism x/j 
of F(X) such that the element ^(< p(u )) does not depend on x n . It is clear 
that the endomorphism ip' = ^oip is injective and rank(</?') = rank(</>) = n. 
Consider the generalized degree function fi given by 

H(xi) = ■ ■ ■ = = 1, n{x n ) = deg (</(«)) + 1. 

We transform the set (p'(X ) = {(p f (xi ), . . . , (p f (x n )} to a //-reduced set 
p"(X) by a finite sequence of elementary transformations. Since ip is injec- 
tive, the set p n {X) consists of n nonzero elements. It is clear that the set 
(p"(X) depends on x n , and the element p'(u) is a polynomial in free gen- 
erators ip n {X) ( F(ip"(X )) = F((p f (X))). Since rank(u) = n, the element 
<p'(u) depends on every free generator from ip n {X). Since the set p"(X) 
depends on x n , by Theorem 11.1.1 we get 

deg(y/(u)) = n(<p'{u)) > n(x n ) = deg (<p'(u)) + 1. 

This contradiction completes the proof. ■ 

The idea of the proof of Theorem 11.5.3 works also for systems of 
elements. Thus we have: 

Theorem 11.5.4. Let X = {x \, . . . ,x n }. Let also U = {ui,...,u m } be 
a system of nonzero elements of the free algebra F(X), rank(C/) = n, ip an 
injective endomorphism of F(X), rank(C7) = rank(</?) = n. Then 

rank({</?(ui), . . . , <p(u m )}) = rank(C/) = n. 

In connection with Theorem 11.5.4, see Theorem 12.7.2 in Section 12.7 
of Chapter 12 about inverse images of primitive systems of elements. 
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Rank Theorems 
and Primitive Elements 



In this chapter, we consider automorphic orbits and combinatorial prop- 
erties of primitive elements in free algebras of the main types of Schreier 
varieties of algebras. A system of elements of a free algebra F is called 
primitive if it is a subset of some set of free generators of F. The rank of 
a system of elements of a free algebra F is the minimal number of generators 
from X on which automorphic images of these elements can depend. 

Let A be a finitely generated free algebra of the variety 9Jl 0 (the variety 
of all algebras), -SUti (the variety of all commutative algebras, char AT 7^ 2), 
or 5 DT 2 (the variety of all anticommutative algebras). 

In Sections 12.1 and 12.2, we consider basic properties of partial deriva- 
tives of elements of A. Some of these properties are based on differential 
separability of subalgebras of A and on the fact that the universal multi- 
plicative enveloping algebra U (A) is a free associative algebra that is a free 
ideal ring. 

Section 12.3 is devoted to elimination of variables. Proposition 12.3.2 
shows that if we have a dependence of derivatives of a homogeneous el- 
ement, then there is an automorphism that eliminates a variable in the 
image of this element. 

In Section 12.4, we obtain the rank theorems. Theorems 12.4.7 and 12.4.8 
show that the rank of a system of elements is equal to the rank of the free 
module generated by the elements whose components are partial deriva- 
tives of elements of this system. Theorems 12.4.9 and 12.4.10 are the rank 
theorems for free Lie algebras and super algebras. 

In Section 12.5, we study primitive systems of elements and actions of 
endomorphisms on automorphic orbits of elements. Theorem 12.5.1 gives 
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a matrix characterization of primitive systems of elements: a system of 
elements of A is primitive if and only if the matrix of partial derivatives 
of elements of the system is left invertible over 17(A). In particular, an 
element of A is primitive if and only if its gradient is unimodular. Based 
on Theorems 12.4.7-12.5.1, we obtain in Theorem 12.5.2 algorithms to find 
the rank of a system of elements and to decide whether this system is 
primitive or not. Theorems 12.5.3-12.5.8 give matrix characterizations of 
primitive systems of elements in free Lie algebras and super algebras. Using 
the Freiheitssatz, we obtain another characterization of primitive elements 
in Theorem 12.5.15: an element a of A is primitive if and only if the factor 
algebra of A by the ideal generated by a is a free algebra in the same variety 
of algebras. 

In Section 12.6, we consider actions of endomorphisms on primitive 
elements. Theorem 12.6.2 shows that endomorphisms of A preserving prim- 
itivity of elements are automorphisms (Theorem 12.6.3 shows that this 
statement is true for free Lie algebras and superalgebras). Using this re- 
sult, we obtain a more general statement in Section 13.3 of Chapter 13. 
Theorem 13.3.3 says that if an endomorphism of A preserves the auto- 
morphic orbit of a nonzero element, then it is an automorphism of A. 
In Theorems 12.6.6-12.6.8 and Problems 12.6.9 and 12.6.10, we consider 
images of nonzero linear combinations of free generators. 

Section 12.7 deals with inverse images of primitive elements of free alge- 
bras of the main types of Schreier varieties of algebras under the action of 
endomorphisms. Theorems 12.7.1 and 12.7.2 show that inverse images of 
primitive systems of elements under the action of injective endomorphisms 
are primitive. In the general case, inverse images of primitive elements 
generate a retract (Theorem 12.7.3). 



12.1 Basic Properties of Partial Derivatives 

Let A be a finitely generated free algebra of the variety 9Jto (the variety of 
all algebras), SPti (the variety of all commutative algebras, char if ^ 2 ), or 
9 Jt 2 (the variety of all anticommutative algebras). 

Let H be a subalgebra of A. By J# we denote the submodule of the 
module I a generated by the elements {T>(h) \ h G H}. A variety 9Jt has 
a property of differential separability for subalgebras if for any a G A we 
have a G H if and only if V(a) G Jh- The varieties 9Jto and OT 2 have the 
property of differential separability for subalgebras. If char AT 7 ^ 2 , then the 
variety 9Jti also has this property. For the details, we refer to [388]. 

In all of what follows, by 9JI we will denote one of the varieties 9Jto, 
(char AT 7 ^ 2 ), and 9 Jt 2 - Also, we use the notations Wi for the linear basis 
of the algebra A and Si for the set of free generators of the algebra U(A). 
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The structure of a U (A)-module on A is given by a • r w = a-w, a-l w = w-a 
(a,w £ A). 

Let H be the subalgebra of A generated by xi , . . . , x n _i, U ( H ) the sub- 
algebra with the unit element of U(A) generated by {r n ,Z n | u £ #}, and 
S H = S fl U ( H ). In the general case, if H is a subalgebra of A , then U(H) 
is not necessarily a subalgebra of U(A). But for Schreier varieties of al- 
gebras, U(H) is a subalgebra of U(A). Let Vh be the left ideal of U(A) 
generated by the elements r u and Z n , where u runs through all monomials 
in # 1 , ... , x n -\ of the degree not less than 1. 



Lemma 12.1.1. For any a £ A and u £ U(H) 

9 1 \ 9 1 \ 

5S (0 “ ) = aS: (0) '"- 

Proof. One may assume that u is a monomial in 5#. Let u = u\ • s, where 
either s = or s — lh, h £ H . Since h £ H, = 0. Hence 



d 

^«°-».) w 



dau\ 


dh 


dau\ 




dx n 


‘ Th \ o * I'aui ~ 

uX n 


dx n 


•rh, 


dh 


daui 


daui 




9x n 


^aui “t" ^ ~ 

uX n 


dx n 



that is, Applying the induction on the 5-degree, we complete 

the proof. ■ 



Lemma 12.1.2. If a £ A and = 0, then a £ H. 
Proof. If = 0, then 

da da 

a ) = yi 1 h y n -i ^ G Jh- 

UX\ OX n —i 



Since the variety 9Jt has the property of differential separability for subal- 
gebras, it follows that a £ H. ■ 

Lemma 12.1.3. If a £ A and £ Vh, then the element a has a pre- 
sentation a = J2 a i u i v i + d , where a* £ K , Ui,Vi £ W , a! £ H, and for 
every i either U{ £ H or V{ £ H . 



Proof. Since algebras A and U (A) and the ideal Vh are homogeneous with 
respect to any Xi £ X, it is enough to prove the statement for a multihomo- 
geneous element a. If a does not depend on x n , then = 0. If a = ax n 
with a ^ 0, then = a ^ V# . Therefore we suppose that a is a multiho- 
mogeneous element of degree not less than 2, and a depends on x n . Then 
a has the unique presentation in the form a = otiUiVi , where u^Vi £ VF, 

oti £ K . 

If = 3 JIq, then 



a = fiwi + f 2 w 2 + . . . fkW k , 



( 12 . 1 ) 
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where ft are linear combinations of words of Wo, Wi E Wo, w\ > • • • > Wk- 
Then 




Since elements of Sh = S fl Vh are left independent over U(A), it fol- 
lows from E Vh that §£^r Wi G Vh- Hence Wi € H or = 0. By 
Lemma 12.1.2, we get either ft G H or Wi G H. 

If 9Jt = 9JTi, then the element a has a presentation in the form (12.1), 
where ft are linear combinations of monomials of W\ less than or equal 
to Wi. Let ft = ctiWi + Qi, 1 < i < fc, where gi are linear combinations of 
monomials less than Wi. Then 




Since w\ > W 2 > • • • > w n , r Qi has no terms of the form a r Wl , a G K. 
Hence, if G Vh , then 



( dwi dgi 

V 011 dx n dx n 



r w i € V H . 



It follows that either w\ 6 H or d ( 2a ^+9i) _ q. that is, either ii'i E II or 
2aiu;i +gi G H. Denote b = a — {2a\Wi +gi)wi. We have b = —a\WiW\ + 



E t *L 2 f% w u ^ e Vh, and 



db 

dx n 



_ dwi 

~ 2ai o r r »"i 




dft dwi 
dx n Vwi dx n Tfi 



E V H - 



Therefore 2a\^ L r Wl G V# . We consider the variety 9Jti over a field K of 
characteristic different from 2. Thus aq = 0, = 0, or w\ G H. The 

condition = 0 is equivalent to W\ G IT, so we get aq = 0 or w\ G IT, 
and it is enough to prove the statement of the lemma for the element 
b + a\W\Wi = X ^=2 fi w i • The induction on fc for the presentation (12.1) 
completes the proof. 

If 9Jt = ®t 2 , then the element a has the presentation (12.1), where ft 
are linear combinations of monomials less than w\. As in the proof for 9Jti, 
it follows from G Vh that -§^r Wl G Vh ; that is, wi G H or ft G iT. 
Considering the element 6 = a — f\W\ and using the induction on the length 
of the presentation (12.1), we complete the proof of the lemma. ■ 




12.2. Homogeneous Admissible Elements 217 



12.2 Homogeneous Admissible Elements 



Let Z+ = {k G Z | k > 0}, Q+ = {a G Q | a > 0}. We say that 
a monomial w € W has the multidegree m(w) = (Aq, . . . , k n ) G Z+ if for 
each i = 1, . . . ,n the generator Xi occurs in w exactly ki times. A linear 
mapping //: Z+ — > Q+ is a functional if pt(s) ^ 0 for all nonzero s G Z™. 
For w G W, we introduce the //-degree by p(w) = (i{m(w)). An element 
a = ^2 i= i Wi G W is said to be //-homogeneous (m-homogeneous) if 
//(wi) = ••• = fi{wk) (m(wi) = ••• = m(wk), respectively). For w G W 
we set m(r^) = m(l w ) = m(w) and fi{r w ) — fi(l w ) = //(w). This gives 
us a possibility to consider //-homogeneous and m- homogeneous elements 
of the universal multiplicative enveloping algebra U(A). The algebra U(A) 
is the free associative algebra with the set S of free generators. Therefore 
elements of U (A) have the 5-degree. 

We say that a set {//i, . . . ,!/&} of //-homogeneous elements of U(H) is 
admissible if it follows from the equation 

r \ k 

UCL v — > . . 

— = (12.2) 

n i = 1 



where a G A, mi, ... , m k G U (A), that there exist ai, . . . , a& G A such that 

k 



da 

dx n 



E dai 



i= 1 



dXn 



Lemma 12.2.1. 



(i) A set with zero elements is an admissible set. 

If {u \, . . . , Uk} is an admissible set , then: 

(ii) The set {ui, . . . , 0} is admissible. 

(iii) The set {u<j(i), . . . , u a ^} is admissible for any element a of the 
symmetric group Sk- 

(iv) IfO^aeK, then the set {ui, . . . , au^} is admissible. 

(v) If u G U(H) and the element Uk + uu^-i is pi-homogeneous , then 
{ui, . . . , Uk~\,Uk + uuk-i} is an admissible set. 



Proof. The statements of points (i)-(iv) are obvious. To prove (v), suppose 
that 



k—i 



k—2 



- — = 'y\m i u i +m k {uk-\-uu k - 1 ) = 'Y.m i u i + (m k -i+m k u)u k -i+m k u k . 
2=1 2=1 

By the conditions of the lemma, there are ...,</& G A such that 

k 



da da<i 

d^ = ^~ Ui ' 



2 = 1 



dx r 
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Therefore 



da 

dx n 




da k 



da k 



-uu k - 1 - - — uu k _ 

UXji C/X n 




dak-i _ dak ' 

dx n dx^ 



Uk - 1 + 



dak 

dx„ 



(u k + uu k - 1 ) 



k — 2 A r\ 

E oai d , 

dx n Ul dx n ak ~ 

1=1 



a k u)u k -i + ^ l (u k + uu k - 1 ). 



Thus the set {ui, . . . , u k + uu k - 1 } is admissible. ■ 

Lemma 12.2.2. Suppose that elements a G A, mi, . . . , m k G U (A), and pi- 
homogeneous elements ui, . . . ,u k G U(H) satisfy the equation (12.2), and 
p(ui) < p{ui) with i > 2. If there is an S-monomial u having a nonzero 
coefficient in the presentation of u\ such that elements U 2 , • • • ,u k have no 
monomials ending on u, then mi e A W- 

Proof. We use the induction on the 5-degree of the monomial u. If u = 1, 
then u\ = au , 0 7 ^ a £ K, and U 2 = • • • = u k = 0 , and 



1 

mi = — 



a 



k 

y = 

i= 1 



1 9a ^ 9 

a dx n dx n v 



It is clear that u G /7(tf). Let u = u 0 s, where s G 5. Consider w € H such 
that either s = r w or s = l w . Since 0 < /x(ui) < /x(ui), i > 2, it follows 
from (12.2) that G V#. By Lemma 12.1.3, a = Yli a i w iVi, where for 
any i either Wi G if" or Vi G if. We write down the element a in the form 

a = aw -w + w- f + g-w + ^^Pj w j v j, 



where / and g are linear combinations of basic monomials different from w 
and Wj , Vj ± w, and for any i either Wj G if or v 3 G if. We get 



da 

dx n 



-®Ll -l =L r + V/3- 

- lw + p) T rw + 2 ^ 



dx n 



dx 



to?. L r +^ll \ 

dx n Vi + dx n “V ' 



The elements have the unique presentation in the form Ui — u's + u”, 
1 <i< k. where elements u" have no monomials ending on s. 

Let s = r w . Since 17(A) is the free algebra on S, comparing in ( 12 . 2 ) 

monomials ending on s, we obtain ^ = Yli=i m i u i- ^ * s eas y to see 
that uo has a nonzero coefficient in u\ and elements u ' 2 ..... u' k have no 
S'- monomials ending on uq. Hence we have proved the statement for uq. 
Since the 5-degree of u 0 is less than the 5-degree of u, by induction we get 

m i e AW- " 

Lemma 12.2.3. Any subset {ui,u 2 , . . . , u k } of ^-homogeneous elements 
ofU(H) is admissible. 
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Proof. We apply the induction on k. Let k = 1. If u\ = 0, then our 
statement follows from Lemma 12.2.1. Suppose that u\ ^ 0, and for some 
a £ A and m i £ U(A) we have = miu\. Consider any 5-monomial u 
that has a nonzero coefficient in u\. By Lemma 12.2.2, mi £ 

Let k > 1. By Lemma 12.2.1, we may assume that fx(ui) < p(ui) with 
i > 2. Consider an 5-monomial u entering in u\ with a nonzero coefficient a. 
One may suppose that a = 1. For the elements u i > 2, we have the 
presentation in the form Ui = u^u^u", where u" have no monomials ending 
on u. By Lemma 12.2.1, the set { u \ , . . . , u k } is admissible if and only if the 
set {^i, U 2 — u' 2 ui , . . . , Uk — u'k u i} is admissible. Since the element u\ is (x- 
homogeneous, the element u\—u has no monomials ending on u. Therefore, 
for i > 2, the element 

Ui — u[ui = u[u + u'l — u\ui = ufa — U\) + u" 

has no monomials ending on u. Hence, we may suppose that elements 
• . . , Uk have no monomials ending on u. By Lemma 12.2.2, m\ = ^ L 
for some a± £ A. Thus 

da dai 

&r = d^ Ui+ 2^ miUi - 

Uds n u ^n 0 

% — A 

Applying Lemma 12.1.1, we get 

da da\ d . . 

flZT - faT u i = ^r( a - 

ujjji iJJsn o 

1=2 

By the induction hypothesis, there are e& 2 , . . . , a* £ A such that 

da dai dai 

dx n dx n Ul “dx n Ut ' 

% — 2 

that is, the set {u±, ...,u k } is admissible. ■ 

Lemma 12.2.4. Suppose that elements ui,U 2 , . . . ,u k ofU(H) are left in- 
dependent over U(A). Then , if for some a £ A, rai,...,rafc £ U(A) the 
equation (12.2) is fulfilled , then mi £ -^-(A), 1 < i < k. 

Proof. By Lemma 12.2.3, there are ai,...,afc of A such that - = 

Y%= 1 Then 52i=i( m i - §^t) u i = Since elements ui,...,u k are 

left independent over U(A ), 1 < i < k. ■ 

12.3 Elimination of Variables 

An endomorphism p of the algebra A is said to be /i-homogeneous if p(xi), 
i = 1, . . . ,n are /^-homogeneous elements, and fx(p(xi)) = fi( Xi ), Xi £ X. 
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The matrix 



«%) = {d(<p{xi)),d((p(x 2 )),...,d{(p(x n ))) 6 M n (U(A)) 

is the Jacobian matrix of the endomorphism p . An endomorphism p is an 
automorphism of A if and only if the matrix J(p) is invertible over U(A) 
([388, 406]). For any fc < n, we set 



/ dcp{x 1 ) 
dxi 


d(p(x k )\ 

dxi 


dtp(x 1 ) 
\ dx k 


dip(x k ) 
dx k ) 



It is clear that J n (p) = J(p)- 

Lemma 12.3.1. Let k be an integer , 1 < k < n, and let p be a p- 
homogeneous automorphism of A such that p(xi) = Xi for all i > k. 
Then the matrix Jk(p) is invertible. If B = ( bij ) = (Jk(p ))~ l , then 
all elements b^ are p-homogeneous and p(bij) = p(xj) — p(xi) (here, if 
p(xj) — p(xi ) < 0, then b^ = 0). 

The proof of this lemma is straightforward. 

The main result of this section is the following proposition: 



Proposition 12.3.2 ([259]). Let a be a p-homogeneous element of A and 
k < n. Suppose that there are fi-homogeneous elements mi, m 2 , . . . ,m k of 
U(A) such that p(mi) = p>(xj) — p,(x n ) for all i < k and 



da da 

dx n ^ dxi ' 

l — L 



(12.3) 



Then there is a p-homogeneous automorphism p of A such that p(xi) = Xi 
for all i > k and p(a) G H, where H is the subalgebra of A generated by 

X\, . . . , X n —\ . 



In order to prove Proposition 12.3.2, at first we prove the following 
lemmas. 



Lemma 12.3.3. Suppose that a positive integer k and an element a of A 
satisfy the conditions of Proposition 12.3.2. If p is a p-homogeneous auto- 
morphism of A such that p(xi) = Xi for all i > k, then k and p(a) also 
satisfy the conditions of Proposition 12.3.2. 

Proof. Since 



fl(y(a)) = y' d M x i)) ( da \ 

dxj ^ dxj \dxi ) 
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and (f(x s ) = x s with s > fc, 



d(v(a)) 

dxj 

d{ip{a)) 



dip(xi) ( da\ . 

EtM — 1. 1 <3<K 



■ , dx i 

7=1 J 



dxi J 



ip 



(£) + S 



d<p(xi) ( da 



dx n r \dx n J ' ^ dx n ^ \dxi 
1=1 

By (12.3), for any ra^, . . . ,m! k G U(A ), we have 



d(<p(a)) ^ , 0(y?(a)) ( da 



dx n 



k 

-E m 

3 - 1 

d(p( x i)) ( ’ Ja 



n 



9^ 



+£ : 

7=1 

k 



dx n 



-ip 



dxi 



E j(E dxj ^\dxi 

7 = 1 N 7=1 J x 



9a \ 



=E^.)^£j + E te 

k / k 

-e(e* 

7=1 V j = l 



7=1 

/ g(y(x i )) V - / da 
j dxj J \dxi 



d (¥>( x i)) ( da 

V \dxi 



vV / \ d(<p(xi)) .9(w(xi))\ / da 

-gH + -^r-g"4 J 5r) 

Consider the system of equations (with unknowns , m(.) 



+ 






dx. 



- = E 



m 



, d((p(xi)) 



■ , J dx, 

J = 1 J 



1 < z < fc. 



We write down this system in the matrix form P = MJk(<p), where 

By Lemma 12.3.1, this system has the unique solution 

9(tp(xi))' 



m ’i = E + 

7=1 ' 



dXn 



bij } 1 ^ j < 



where B = (Jk((f)) 1 = ( bij ). Remark that 



M m j) = vivim) ■ hj ) = /x 



dx n 



b, 



13 



n{xi) - n(x n ) + n(xj) - n(xi) = fj,(xj) - fj,( 
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It follows from = Y^= 1 m j that ^ an< ^ T( a ) sa ti s fy the con- 
ditions of Proposition 12.3.2. ■ 

Consider the following decompositions of algebras A and U(A ): 

A = A 0 © Ai © • • • ® A k © . . . , 

U (A) = i/o 0 E/i ® • • • 0 i7fc 0 . . . , 

where Ai and Ui are subspaces of all homogeneous elements with respect 
to x n with the x n -degree equal to i, A 0 = H, Uo = U ( H ). For an element / 
of A (of U (A), respectively), by / W we denote its component of Ak (of £/*, 
respectively). 

Lemma 12.3.4. Suppose that a positive integer k and an element a of A 
satisfy the conditions of Proposition 12.3.2, and there is a positive inte- 
ger s, 1 < s < k, such that the elements ^-(a^ 0 )), . . . , ^f-(a^) are left 
independent and dx d s+1 (a^) = • • • = ^f^( a ^) = 0. Then, for any r > 0, 
the following conditions are equivalent: 

(i) a ^ = a ^ = • • • = a^ = 0; 

(ii) = 0 for all i = 1,2, ... , s and j = 0, 1, . . . , r — 1. 

Proof. It follows from (12.3) that 



dx 



%«'«>)= 

t=i 



\dx r 
k j 



da\ U) 

dxi ) 



K J / n \ (j — 0 k j 

EE"*"’ (§t) =EE (12.4) 

i=lZ=0 V i=l 1=0 ox% 



for all j = 0, 1, . . . ,r — 1. 

If (i) is fulfilled, then we rewrite (12.4) in the form 



E"*“£-( o<0) ) = E m “£-(“ (0> ) = »• 



i = 1 



i=l 



Bearing in mind that elements ^f-(a^) ? 1 < i < s, are left independent, 
we get (ii). 

If (ii) is fulfilled, then we write down (12.4) in the form 



d 

dx n 



(« 0 + 1) )= S Y^ m i l) 



t=s+l 1=0 



d_ 

dxi 



(a^>). 



If j — 0, then = 0. By Lemma 12.1.2, G H = A$\ that is, 

a- 1) = 0. Suppose that for some t, t < r, we have a^ 1> = a < ' 2> = • • • = 
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a M — o. Then, setting j = t in the previous equality, we get 
n <=s+l 1=0 * 



Hence = 0, and the induction on t completes the proof. ■ 

Lemma 12.3.5. Suppose that all conditions of Lemma 12.3.4 are ful- 
filled. If a W = a ® = • • • = a ^ = 0 for some r > 0, then there is 
a p-homogeneous automorphism <p of A such that 

<p(xi) = Xi with x > fc, ((p(a))^ = a^°\ 

(¥>(<0) (1) = (¥>(«)) (2) = • • • = (<^(a)) (r) = (^(a)) (r+1) = 0. 

Proof. It follows from (12.3) and (12.4) that 






EE> 

i = 1 1=0 



(0 



dx t 



-(a^- *>) 



i — 1 i = 1 

Since elements ^(a^ 0 )), . . . , ^(a^) of U(H) are left independent and fi- 

homogeneous, by Lemma 12.2.3 there are elements gi of A such that = 
Jjp-, 1 < i < s. One may assume that the elements gi are //-homogeneous 
and gi G A r + 1 , 1 < i < s. Then 



Mfli) = l J -( m i r) ) + t J '( x n) = li(mi) + n(x n ) = f. t{xi)-n(x n ) + n(x n ) = n{Xi). 
Consider the homogeneous endomorphism of A given by 



<p(x%) = 



9ii i — ^5 



Without loss of generality, we may assume that //(#i) > p(x 2 ) > ••• > 
/ i(x s ). Since for j < i < s 

K m i) = K X i) ~ K X n) < Ufa) < ll(Xj), 

the element mi does not depend on xj. It follows from m\ r ^ = and 
gi G A r +i that gi does not depend on Xj. Since g^ G A r +i and r + 1 > 0, 

, N da da da 

<p(o) -° = - 91 g ‘ te . +b ' 

where b G A 2 r +2 0 ^ 2 r +3 0 . . .. Moreover, 

((p(a))® = a^ l \ 0 < / < r, 



(<^(a)) ( 



T+l) _ n( r+1 ) 






i= 1 
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Therefore 



(</?(a)) (r+1) = a (r+1) - 

1=1 1 



Since ^-(a (0) ) €U(H), 



dx , 



-(M«)) (r+1) ) = ^(a (r+1) )-E 



i = 1 



% 

dx n dx, 



(«,<»>) 



8 i« i '«)-E»r^ ( » i,, )=». 

i=l 1 



dx n 



Lemma 12.3.6. Suppose that all conditions of Lemma 12.3.4 are fulfilled. 
Then there is a fi-homogeneous automorphism <p of A such that p(a) G H 
and p(xi) = Xi with i > k. 

Proof. Using the induction on r (with the base r = 0) and Lemmas 12.3.5 
and 12.3.3, for any r we find an automorphism p r with p(xi) = Xi for 
i > fc, (y? r (a))(°) = a(°\ (y? r (a))^ = ••• = (yv(a))^ = 0. Take r such 
that r + 1 > If for some s > r we have (p r (a))^ ^ 0, then 

p(x n ) • 5 < p(p r (a)) = //(a); 

thus r + 1 < 8 < Jgy. This contradiction shows that p r is a desirable 
automorphism. ■ 

Proof of Proposition 12.3.2. We use the induction on k. If k = 0, then 
= 0; that is, a G H, and for p we may take the identical automorphism 
of A. Suppose that k > 0 and the statement is proved for all k' < k. We 
denote b = a^°\ If the elements . . . , ^ are left dependent, then 

by the weak algorithm (Theorem 11.2.1), without loss of generality, we may 
suppose that there are elements m^, . . . of U(A) such that = 

By fh e induction hypothesis, there exists a //-homogeneous 
automorphism ^ with ^~(V ; (^)) = 0 and ip(xi) = Xi for all i > k. Therefore 
i/;(b) = (V>( a ))®. By Lemma 12.3.3, the integer k and the element ip(a) 
satisfy the conditions of Proposition 12.3.2. Changing the element a by 
if (a) and applying the arguments above, we may assume that there is an 
integer s < k such that the elements ^ , . . . , are left independent and 

a(q(°>) _ = d{gW) = 

dx s +i dx k 

Applying Lemma 12.3.6, we complete the proof. ■ 
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Following [352], [354], we define the rank of an element a of a free alge- 
bra A (rank(a)) as the minimal number of generators from X on which 
an automorphic image of a can depend. The rank of a system of elements 
ai, . . . , afc of A is defined similarly (notation: rank(ai, . . . , a^)). 

For an element a of A, by M a we denote the left U( A)-submodule of 
U(A) generated by the elements . . . J—. The universal multiplicative 
enveloping algebra U(A) is a free associative algebra. The algebra U(A) as 
a left [/(A)-module is a free one-generated module. Any left ideal of a free 
associative algebra is a free module of unique rank. By rank(M a ) we denote 
the rank of the module M a . 

Lemma 12.4.1. Let (p be an automorphism of A. Then rank(M^( a )) = 
rank(M a ). 

Proof. 



fl(y(o)) 

dxj 



^ d(ip(xj)) ( da\ 
^ dxj * [dxj 




d(<f{xj)) 

dxj 



da 

dxi 



G V>(M 0 ); 



that is, M^( a ) C ip(M a ). Therefore 

ip{Ma) = c = M^(a) . 

Thus (p(M a ) = Af v ( a ) and rank (M^) = rank(M a ). ■ 

Lemma 12.4.2. If a £ A, then rank(M a ) < rank(a). 

Proof. Let k = rank(a). Then there is an automorphism <p> of A such that 
the element <p(a) belongs to the subalgebra generated by a?i, X 2 , . . . , 
and the left [/(A)-module is generated by , . . . , . Thus 

rank(M a ) = rank(M^( a )) < k. ■ 

Let /x: Z” — >• Q+ be a functional. For a > 0, by A a ( U a ) we denote the 
subspace of /i-homogeneous elements of A (of U(A), respectively) with the 
fi - degree equal to a, 

A=($A a , U{A)=®U°. 

ot GQ+ a€Q+ 



For f £ A(f £ U(A)), by p“(/) we denote the /^-homogeneous component 
of /, />“(/) € A a ( //" ( / ) £ U a , respectively). Following [272], we say that 
an element a of A is //-bounded if p“(a) = 0 for any a > 1. 

Let V(p) = {m £ Z” | /i(m) < 1}. Consider the following par- 
tial ordering on the set of functionals: /ii -< /t 2 if F(mi) c V(// 2 ) and 

V(ni) ± F( M2 ). 
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Lemma 12.4.3. Let a be a p-homogeneous element of A and rank (M a ) < n. 
Then there exists a p-homogeneous automorphism ip of A such that the 
element depends on not more than n — 1 variables of X. 

Proof. If ppitpia)) depends on not more than n— 1 variables of X, then 
for p we take the identical automorphism. 

Suppose that /^(^( a )) depends on every variable of X. Since rank(M a ) < 
n, there are not all equal to zero elements mi, . . . m n of U(A) with 

71 Q 

E ua 

mu— = 0. (12.5) 

i = l OXl 

For i — 1, . . . , n, we set 

«* = max{a € Q + | p“(m t ) ± 0 }, 
j3 = max{aj - p(xi) + 1 | i — 1 , . . . , n} 

(if mi = 0, then a* = — 1 ). Since p* (a) depends on x \, . . . , x n , l — p(xi) > 0 
for i = 1, . . . , n, and /3 > 0. Applying p^ to (12.5), we get 



*=1 a€Q+ 



= E E /^K) • ^pT KXi] ^)) = ( 12 - 6 ) 

i= 1 o:€Q+ * 

If a+p(xi) > 1 , then taking into account that a is a //-bounded element, we 
get p^^ Xi \a) = 0. If a + p,(xi) < 1 , then /? — a > a* — p(xi) + 1 — a > ap, 
that is, = 0. Therefore, it follows from (12.6) that 



Since for some i 



P - 1 + p(x{) = (a* - p(xi) + 1) - 1 - p(xi) = a< > - 1 

and = /^(m*) 7 ^ 0 , it follows that elements -^f-(/^(a), 1 < 

i < n, are left dependent. Since these elements are //-homogeneous, there is 
j, 1 < j < n, such that the element ^-(/^(a)) belongs to the left ideal of 

U(A) generated by elements ^-(/>^(a)) with i 7 ^ j. By Proposition 12 . 3 . 2 , 
there exists a //-homogeneous automorphism tp of A such that <£>(pjUa)) 
does not depend on xj. Now, to complete the proof it is enough to note 
that (p(p%(a)) = p*{p{a)) for any a G Q+. ■ 

Lemma 12.4.4. Let a be a p-homogeneous element of A. Suppose that a 
depends on #i, . . . , x n . Then there exists afunctional po such that V(po) C 
V(p), a is po-bounded, and the component p* o (a) depends on x 1 , . . . ,x n . 
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Proof. Suppose that p* (a) depends only on x \, . . . , Xk (k < n). To prove 
the lemma, it is sufficient to find a functional po such that V(po) C V(p ), 
a is //o-bounded, and the component p* o (a) depends onxi,... ,Xk,Xk+i- 
For any t G Q+, let p t be the functional given by pt{xi) = oti with 
i ^ k + 1, and pt(xk+ i) = //(£fc+i) + t. It is clear that V(p t ) C V(p)- Since 
p* (a) does not depend on ajfc+i, Pt(p^( a )) — 1- The element a is //-bounded 
if for any monomial w having a nonzero coefficient in a such that it has 
the Xfc+i-degree equal to r > 0, we have p(w) + rt < 1. Since p* (a) does 
not depend on Xk+i, p{w) < 1. Hence, for the element a, the property to 
be //-bounded is equivalent to a system of inequalities of the form t < a* 
(cti > 0). Let to be a maximal solution of this system. Then there exists 
a monomial w of the degree r > 0 with respect to Xk+i such that Pt Q (w) = 
fi(w)+rto = 1. Therefore the component p * (a) depends on Xk+i- It follows 
from p>t 0 (p)i{a)) = 1 that p* (a) depends on x\, . . . , £&, ^/c+i- Finally, we set 
= ■ 

Lemma 12.4.5. Let p be a functional , and let a be a p-bounded element 
of A. Suppose thatr&nk(M a ) < n. Then there exist an automorphism <p of A 
and a functional po such that po -< p and the element (p(a) is po-bounded. 

Proof. By Lemma 12.4.4, we may assume that p*(a) depends on all 
variables of X. By Lemma 12.4.3, there is a //-homogeneous automor- 
phism p of A such that p^(<p(a)) depends at most on n — 1 variables. 
We may suppose that p* (<p(a)) does not depend on x n . Since p* (a) de- 
pends on x n , there exists a monomial w of the x n -degree equal to r > 0 
having a nonzero coefficient in a. For t G Q+, we consider the functional p t 
given by pt{xi) = Xi with i ± n and pt{x n ) = p(x n ) + 1. As in the proof of 
Lemma 12.4.4, we find to > 0 such that <p(a) is a //^-bounded element. We 
set po = //t 0 - It is obvious that V(po) Q V(p). From p(w) = 1, it follows 
that po(w) = p(w) + rto > 1. Therefore m(w) G V(p), but m(w) £ V(po)\ 
that is, po -< p. ■ 

Lemma 12.4.6. Let a G A, rank(M a ) < n. Then, there is an automor- 
phism cp of A such that <p(a) G H, where H is the subalgebra of A generated 
by X\ , . . . , x n —i . 

Proof. Let d be the degree of a. We set p(xi) = i = 1, . . . , n. Then a 
is a //-bounded element. We use the induction on the cardinality \V(p)\ of 
the set V(p). If | V(p)\ = 1, then V(p) = {(0, . . . ,0)}, and a = a • 1 G H 
(a G K). Suppose that the statement of the lemma is proved for any // 0 - 
bounded element a (po -< //). Applying Lemma 12.4.5, we complete the 
proof. ■ 

Theorem 12.4.7 ([259]). For any element a of A, the rank of a is equal 
to the rank of the left ideal M a of U(A) generated by the elements 
1 < i < n. 
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Proof. By Lemma 12.4.2, rank(M a ) < rank(a). Let k = rank(a). We may 
assume that a belongs to the subalgebra S of A generated by #i, . . . , Xk- By 
Lemma 12.4.6, the elements Jj_, 1 <i < k, are left independent over U(S). 
Since U (A) is a free U (S')-module, these elements are left independent over 
U{A). Therefore rank(M a ) = k. ■ 

Let ei {i = 1, . . . , r) be the row with ith coordinate equal to 1 and other 
coordinates equal to 0. 



Theorem 12.4.8 ([259]). The rank of a system of elements a \ , . . . , a r of A 
is equal to the rank of the left U(A)-module M = generated by 

the elements Y^i=i 1 — 3 — n - 

Proof. Let s = rank(M{ ai? _ >ar }), k = rank({ai, • • • , a r })* As in the proof 
of Lemmas 12.4.1 and 12.4.2, we see that s = rank(M{ y ,( 0l ) v .. j¥? ( ar )}) for 
any automorphism <p of A , and k > s. Therefore we may assume that the 
elements a* (1 < i < r) belong to the subalgebra generated by #i, . . . , 

Let Y = {yj \ 1 < j < r}. Consider the free algebra B = Fgjt(X U Y). 
It is clear that A C B and U(B) is a free right U(A)- module. Therefore 
s = rank[/(^) U(B)M. Consider the element 

b = a\yi + a^y 2 + • * • + o> r y r . 



It is obvious that rank(6) < k -f r, 



db_ 

dxj 



E OOi 

dxj 

l=L J 



r Vi , 



1 <j<n, 



dy s ae ’ 



1 < s < r. 



We may suppose that the elements ai,...,a r are linearly independent. 
Then the elements r yi , . . . , r Vr , l ai , . . . , l ar are left independent over U ( B ). 
Hence 



M b 




1 < j < n 




© • • * ® U(B)l ar . 



The homomorphism / of U(B) given by /(e^) = r Vi induces the 
isomorphism 




1 < j < n 



| s U(B)M. 



Thus, rank^(^) M b = s + r. By Theorem 12.4.7, rank(6) = s + r<fc + r; 
that is, s = k. ■ 



Now we consider rank theorems for free Lie algebras and superalgebras. 
In all of what follows, we denote Lx = L(X ) (the free color Lie superal- 
gebra) in the case where charRT = 0, and Lx = L P (X) (the free color Lie 
p-super algebra) in the case where char K = p > 2. 
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Theorem 12.4.9 ([272, 273]). Let h be a G -homogeneous element of Lx - 
Then rank(/i) is equal to the rank of the left ideal of K(X) generated by the 
elements {|| | x G X} (as a free left K(X) -module). 

Theorem 12.4.10 ([272, 273]). Let hi , . . . , hk be G -homogeneous ele- 
ments of Lx- Then the rank of the system {hi , . . . ,hk} is equal to the rank 
of the left K(X) -submodule of the K(X) -module 

K(X) k = K(X)ei © • • • © K(X)e k 

generated by the elements 




xeX 



} 



As was proved by U. U. Umirbaev in [391], the statements of Theorems 
12.4.9 and 12.4.10 take place for the free group F (instead of K(X) it is 
necessary to consider the group algebra of F over a field K ; this algebra is 
a fir). 



Problem 12.4.11. Find criteria for an element of a polynomial or a free 
associative algebra to have a given rank and an algorithm to determine the 
rank of an element. 



Theorems 12.4.7-12.4.10 (together with the weak algorithm) give us an 
effective algorithm to find the rank of a system of elements of a free algebra 
of a Schreier variety of algebras. For a homogeneous element of a free Lie 
algebra, an algorithm to find the rank was constructed by V. Shpilrain [354]. 

A. A. Mikhalev and A. A. Zolotykh [272, 275, 280] obtained algorithms 
to realize the rank of a system of elements in free Lie (p-) (super) algebras 
(i.e., algorithms to construct a concrete automorphism such that the im- 
ages of elements depend on a minimal possible number of free generators) . 
For free ((anti) commutative) nonassociative algebras, such algorithms were 
constructed by K. Champagnier [66]. 



12.5 Primitive Elements 

We recall that a system of elements of a free algebra F is said to be primitive 
if it is a subset of some set of free generators of F. 

Let X = {xi, . . . ,x n }. Let also A be the free algebra with the set X 
of free generators of the variety 9Jt 0 (the variety of all algebras), 9Jti (the 
variety of all commutative algebras, char if ^ 2), or 9Jt2 (the variety of all 
anticommutative algebras) . 
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12.5.1 Primitive elements in free nonassociative algebras 

Theorem 12 . 5.1 ([ 259 ]). A system a \ , . . . , a r of elements of A is primi- 
tive if and only if the matrix (<9(ai), . . . , d(a r )) is left invertible over U(A). 
In particular, an element a of A is primitive if and only if there are 
mi, ...,m n eU{A) such that Yft=i = 1- 

Proof. Let e* (i = be the row with ith coordinate equal to 1 

and other coordinates equal to 0. By M{ 01? ... ?0r } we denote the left U(A)~ 
module generated by the elements YH=i 1 < 3 < n - If the matrix 

(d(ai ), . . . , d(a r )) is left invertible over U(A), then 

r 

M {0l ,„. i0r} = U(A) ei ® U(A)e 2 ® • • • ® U(A)e r = ® U(A) ei . (12.7) 

i—\ 

By Theorem 12.4.8, rank(ai, . . . ,a r ) = r. Moreover, it follows from (12.7) 
that for any automorphism p of A we have 

r 

%(<»l) VK)} = ®U{A)e, 

i — 1 

We may assume that ai, . . . , a r belong to the subalgebra S of A generated 
by xi, ... ,x r . Let xjj be an endomorphism of S given by ^(xi) = a *. It 
follows from (12.7) that the Jacobian matrix of xjj is invertible. Therefore 
ip is an automorphism. The inverse statement is obvious. ■ 

Theorem 12 . 5.2 ([ 259 ]). For a system of elements ai, . . . , a r of A, there 
is an algorithm to decide whether this system is primitive or not. 

Proof. For one element a of A , we consider the left ideal M a of A. Using 
elementary reductions, we construct the standard Grobner-Shirshov basis 
of this ideal. The cardinality of this basis is exactly the rank of a. The 
element a is primitive if and only if rank(a) = 1 and the standard Grobner- 
Shirshov basis of M a consists only of the unity element. 

In the general case, the rank of the system a \ , . . . , is equal to the left 
rank of the matrix ( d(a \ ), . . . ,9(a^)). Using (left) elementary transforma- 
tions of rows, we find this rank. The system is primitive if and only if this 
matrix is left invertible. ■ 



12.5.2 Primitive elements in free Lie superalgebras 

Now we consider primitive systems of elements of free color Lie (p-)super- 
algebras. 

Let L be the free color Lie superalgebra L(X) or the free color Lie p- 
superalgebra L P (X). For a G L we denote by d(a) = dx(a ) the column 
( JzT’ • * * ’ • ^ or an endomorphism p of L by J/e (</?), k <n, we consider 
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the (k x fc)-matrix 



^ki}P) — 



/ d< f( x i) 
1 dx X 



. dip(x i) 

\ dx k 



d(f(x k ) \ 
dx i 1 



dy(x k ) I 

dx k ) 



and we say that Jk{p) is the fcth Jacobian matrix of p. If k = n , then 
J(p) = J n (p) is called the Jacobian matrix of p. 

The following theorem gives a positive solution of the Jacobian problem 
for free color Lie (p-) super algebras. 



Theorem 12.5.3 ([249]). Let K be a field and X = (aq, . . . , x n }. An en- 
domorphism p of the free color Lie (p-) superalgebra L is an automorphism 
if and only if the Jacobian matrix J(p) is invertible over A(X). Moreover , 
J(p)-' = p(J(p-')). 



Note that a square matrix over a free associative algebra is left in- 
vertible if and only if it is right invertible. It follows from the fact that 
a free associative algebra can be embedded into a skew field (see the 
monographs [80, 81, 328]). The Jacobian conjecture for free Lie alge- 
bras was proved by U. U. Umirbaev [387], C. Reutenauer [312], and 
V. Shpilrain [351]. More general situations for Lie algebras were considered 
by U. U. Umirbaev [387] and V. Shpilrain [351]. The Jacobian conjec- 
ture for free Lie (p-) super algebras was proved by A. A. Mikhalev [249]. 
The Jacobian conjecture for free Lie algebras over rings was proved by 
A. A. Mikhalev and A. A. Zolotykh [274]. The Jacobian conjecture for 
free nonassociative algebras and free (anti)commutative (nonassociative) 
algebras was proved by A. V. Yagzhev in [406] and for other types of free 
algebras with the Nielsen-Schreier property by U. U. Umirbaev [388]. 

Note that for the composition of endomorphisms p and ^ of L(X) (of 
L P (X)), the Jacobian matrix o p) is equal to J(^) , ip(J(p)). 

It is possible also to give a solution of the Jacobian problem for Lie 
superalgebras in the following form (the exposition of this result for Lie 
algebras was given by U. U. Umirbaev): 

Theorem 12.5.4. Let X = {x u ...,x n }, Y = {y u . . . , y n }, d{xi) = d(yi), 
xnr = 0,L = L(X) (L = L p {X)), V = L(XUY) {V = L P (XUY)), and 
let p be an endomorphism of L, p{xf) — fa. Consider the endomorphism p ' 
of L' given by 

v'&i) = fi <P f (yi) = 1 <i<n, 

where f[ are X -linear parts of the elements 



fi I zi=zi-H/i > i — 1, . . . , n. 

—Eri ~\~yn 

Then ip' is an automorphism of 1! if and only if <p is an automorphism 
ofL. 
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Theorem 12.5.3 gives an effective procedure to recognize automorphisms 
of free Lie algebras and superalgebras among endomorphisms. Another 
algorithm follows from Theorem 11.1.1. 

Let X = {xi, . . . , x n }. For the free associative algebra K(X), denote by 
K(X) e the tensor product K{X)®kK{X) with the multiplication given by 
(a®5)(c®d) = ac<g>db. It is clear that K{X) e is the universal multiplicative 
enveloping algebra for K{X). Let 

I a = ®(K(X) e )i = ©(#<*> • dxt ■ K (X))i 

i— 1 i= 1 

be the direct sum of n copies of K(X ) e , ( K(X) e )i = K(X) • dxi • K{X), 
(a®b)i = a-dxi-b. The universal derivation of K{X) is the AT-linear mapping 
D : K(X) I a given by D(xi) = 1 • dxi • 1, D(ab) = D(a) • b + a • D(b), 

D( 1) = 0. For any a G K{X), the element D(a) has the unique presentation 

in the form D(a) = a\ H 1- a n , a>i G ( K(X) e )i . The component ai = ^ 

is the partial derivative of a on Xi. 

If we consider the natural homomorphism of associative algebras with 
the identity elements <p: K(X) e — )> K[X] (where K[X] is the polynomial 
algebra), (p(a ® b) = a6, then the images of partial derivatives of elements 
of K(X) are the usual partial derivatives of elements of K[X], 

If X = {# 1 , . . . ,x n } and <p is an endomorphism of K(X), then (p is an 
automorphism if and only if the Jacobian matrix 




is invertible over K(X) e . This result for n = 2 was obtained by 
W. Dicks and J. Lewin [96] and for arbitrary n by A. H. Schofield [328]. 
A. A. Mikhalev and A. A. Zolotykh proved this result for free associative 
algebras over an associative commutative ring [276]. 

The statement of the Jacobian conjecture fails for K(X) if we use Lie 
partial derivatives. For example, let X = {^ 1 ,^ 2 }, 1 ) = x\ + # 2 X 1 X 2 , 

<p(x 2 ) = X 2 - Then 




is invertible, but at the same time ip is not an automorphism. But in the 
case n — 2, V. Shpilrain [350], using Dicks’ commutator test, proved that 
ip is an automorphism if and only if the matrix 

/ M*i)s I t 

is a nonzero scalar matrix. 

The following results give matrix criteria for a system of elements of free 
Lie (p-)superalgebras to be primitive. 




12.5. Primitive Elements 233 



Theorem 12.5.5 ([272, 273]). Let h be a G -homogeneous element of Lx - 
Then the following conditions are equivalent: 

(i) The element h is primitive. 

(ii) There exist elements mi, ... , m n e K(X ) such that 



E 




= 1 . 



Note that the statement of Theorem 12.5.5 does not take place for free Lie 
algebras over fields of positive characteristic. A. A. Mikhalev, U. U. Umir- 
baev, and A. A. Zolotykh constructed a series of such examples in [261, 262]. 
We show one such example. Let K be a field, char K = p > 2, X = {x, y, z }, 
L(X) the free Lie algebra, and h = x + [y, z] + (adx) p (z) G L(X). Then 
h is not a primitive element of L(X), but at the same time the rank of 
the left ideal of K(X) generated by the elements is equal 

to 1 (ft is a primitive element of L P (X) but not of L(X)). Indeed, sup- 
pose that ft is a primitive element of L(X). Then there exist primitive 
elements /i, / 2 of L(X) such that {/i,/2,ft} is a primitive system of el- 
ements, L(X) = L({/i,/ 2 ,ft}). By Theorem 11.1.1, we may assume that 
£{fi) < ^(/2) < ^(ft) and that the leading part ft° of ft is a Lie polynomial on 
leading parts /°, /| of elements /i, / 2 , respectively. Since ft° = (ad x) p (z), 
we may assume /° = ax, 0 ^ a E K. Therefore fi = ax, and 

h--h = [y, z] + (adz)^) 
a 



is a primitive element. It gives a contradiction with the fact that any set 
of free generators of L(X) forms a basis of L(X) modulo [L(X),L(X)]. 
Consider now the partial derivatives of ft: 



dh y dh dh 

sT = 1 + x z ’ TT = 
ox oy oz 



-y-x 



v 



Setting m\ — 1, m 2 = —x p x , m3 = 0, we get the equality 



dh dh dh 
mi— +m 2 — + m 3 — 
ox oy oz 



1. 



Let I be the ideal of L(X) generated by ft, L = L(X)/I. Since ft is not 
a primitive element of L(X), L is not a free Lie algebra ([191]; see also 
Section 12.5.3). But the universal enveloping algebra of L is a free associa- 
tive algebra of rank 2. Indeed, U(L) = K(X)/J , where J is the ideal of 
K(X) generated by ft. At the same time ft is a primitive element of K(X): 
ft = x + [y + x p , z\, it can be obtained from x by applying two elementary 
automorphisms. In particular, the cohomological dimension of the Lie al- 
gebra L is equal to 1 [261, 262, 275]. It shows that there is no analog of 
the Stallings-Swan theorem (that groups of cohomological dimension 1 are 
free [372, 381]) for free Lie algebras over fields of prime characteristic. 
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Problem 12.5.6. Let char if = 0. Is it true that a Lie if-algebra L is 
free if and only if its universal enveloping algebra U(L) is a free associative 
if-algebra? 

Problem 12.5.7. Let char K = 0. Is it true that a Lie if-algebra L is free 
if and only if its cohomological dimension is equal to 1? 

This problem was solved in the affirmative for two-generated Lie algebras 
by G. L. Feldman [127]. Some necessary and sufficient conditions for Lie 
algebras with one defining relation to have cohomological dimension 1 were 
obtained by A. V. Sereguine [333]. 

Theorem 12.5.8 ([272, 273]). Let hi , . . . , hk be G -homogeneous elements 
of Lx • Then the following conditions are equivalent: 

(i) The system {fti, . . . , hk} is primitive. 

(ii) There exist elements mij E K(X), i = 1, . . . , k, j = 1, . . . , n, such 
that for all r, s = 1, . . . , k 

dh s _ 

nflri r\ — Ors) 

OXi 

i.e., the ( n x k) -matrix (§^-) is left invertible. 

Theorems 12.5.5 and 12.5.8 give us an effective algorithm to decide 
whether a system of elements of a free color Lie (p-superalgebra) is primitive 
or not. Moreover, A. A. Mikhalev and A. A. Zolotykh [272, 275, 278, 280] 
constructed an algorithm to construct a complement of a primitive sys- 
tem of elements with respect to a free generating set of the free Lie 
(p-)superalgebra. Based on Theorem 12.5.1, K. Champagnier [66] con- 
structed a similar algorithm for free ((anti) commutative) nonassociative 
algebras. 

Problem 12.5.9. Are the assertions of Theorems 12.5.5 and 12.5.8 valid 
for free Lie rings? 

Problem 12.5.10. Find criteria for an element of a polynomial or a free 
associative algebra to be primitive and an algorithm to determine primitive 
elements. 

V. Shpilrain and J.-T. Yu [359, 360] constructed algorithms to recognize 
coordinate polynomials of a polynomial algebra of rank 2 (primitive ele- 
ments of a free associative algebra of rank 2, respectively) over a field of 
characteristic zero; see Chapter 9. 

Problem 12.5.11. Is it true that an element a of the free associative 
algebra K(x,y) is primitive if and only if there are elements 61,62 of the 
algebra K(x, y) e such that 61 + 62 = 10 1? 
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Theorem 12.5.1 for a single primitive element of a free group was proved 
by W. Dicks and M. J. Dunwoody [95, Corollary IV.5.3], and for a system 
of elements of a free group a result similar to Theorem 12.5.8 was obtained 
by U. U. Umirbaev [389]. An algorithm to recognize primitive elements of 
free groups was constructed by J. H. C. Whitehead in [399, 400] (in fact, 
he solved a more general automorphic conjugacy problem; see [227]). 



12.5.3 The Freiheitssatz and primitive elements 

In this section, Theorem 12.5.15 gives a nonmatrix criterion for an element 
to be primitive. 

Proposition 12.5.12. Let I be a nonzero ideal of A such that the fac- 
tor algebra A/ 1 is a free algebra of the variety 971 with k free generators. 
Then there is a primitive system of elements g \, . . . ,g n -k of A such that it 
generates the ideal I. 

Proof. Let cp: A — » B = Fgyt(yi, ...,?/&) be a surjective homomorphism, 
ker(<p) = I. We set fi = <p(xi), 1 < i < n. Consider the system of elements 
/i> /2? • • • > /n of the algebra B. Since the variety 971 has the Nielsen prop- 
erty (Theorem 11.1.1), by a finite sequence of elementary transformations 
we can transform this system to yi, . . . , 0, . . . , 0. Applying the same se- 

quence of transformations to xi, . . . , x n , we get a new set of free generators 
Pi, • • • ,Pfc, qi, • • . , q n -k of A. It is clear that (p(pi) = <p(qj) = 0 for all 
i, j. Thus the ideal I is generated by q\, . . . , q n -k- ■ 

An algebra S defined by n generators and m relations with n > m in 
a fixed variety is said to satisfy Lyndon’s condition if among given genera- 
tors there are n — m elements generating a free subalgebra in this variety. 
O. G. Kharlampovich [175] proved that free algebras of the variety of all 
Lie algebras and of the variety of all /-step solvable Lie algebras satisfy 
Lyndon’s condition. 

Theorem 12.5.13 (Freiheitssatz, [347, 416]). Suppose that an ele- 
ment a of A depends on a free generator x £ X. Let (a) be the ideal of A 
generated by (a). Then the setX\x generates a free algebra of the variety 971 
in the factor algebra A/ (a). In other words, F(X \x)D (a) = 0. 

The Freiheitssatz in group theory is due to W. Magnus [229]. For free 
nonassociative algebras, the Freiheitssatz was proved by A. I. Zhukov [416] 
and for free commutative and anticommutative nonassociative algebras by 
A. I. Shirshov [347]. For Lie algebras, this result was obtained by A. I. Shir- 
shov [348]. In associative algebras, these problems were solved in the case 
of the field of zero characteristic by L. G. Makar-Limanov [234] and in the 
case of a homogeneous relation by M. Hedges in [150]. We mention also 
papers [92], [141], and [215] devoted to the study of associative algebras 
with one defining relation. 
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An element u of a free group F is a root of an element w E F if w 
belongs to the normal closure of u in F. W. Magnus [229] showed that if 
w is primitive, then (to within conjugacy and inversion) the only root of w 
is the trivial one, namely w itself. By analogy, if u,w are nonzero elements 
of a free algebra F(X ), then we say that u is a root of an element w G F 
if w belongs to the ideal of F(X) generated by u. Let F(X) be a free 
((anti)commutative) nonassociative algebra, a free Lie algebra over a field 
of zero characteristic. Suppose that w is a primitive element of F(X). Then 
it follows from the Freiheitssatz that all roots of w are trivial (i.e., of the 
form aw, 0 ^ a € K). Note that if the field K is constructive and has an 
algorithm for square root extraction, then there is an algorithm to decide 
whether an even element of a free color Lie superalgebra is a square of some 
odd element and, if so, to find a square root [33, Chapter 2, Theorem 2.14]. 

For more facts on the application of the Freiheitssatz, we refer to the 
review article [178]. 

Problem 12.5.14. Does the Freiheitssatz take place for Lie p-algebras 
and color Lie p-superalgebras? 

One can find some particular examples of positive answers to the 
Freiheitssatz for free Lie (p-)superalgebras in [33, 246, 251, 275]. 

Theorem 12.5.15. Let a be a nonzero element of A, (a) the ideal of A 
generated by (a). Then a is a primitive element if and only if the factor 
algebra A/ (a) is a free algebra of the variety 9Jt. 

Proof. Let A/I = F<m{yi, . . . , yk)- By Proposition 12.5.12, we may assume 
that the ideal (a) is generated by x\, . . . , x n -k . By Theorem 12.5.13, n—k = 
1, and a = axi, 0 ^ a G F. 

If a is a primitive element of A, then obviously A/ (a) is a free algebra of 
the variety 3DT. ■ 

The result similar to Theorem 12.5.15 was proved for free Lie algebras by 
G. P. Kukin in [191] and for two-variable polynomial algebras over a field 
of zero characteristic by S. S. Abhyankar and T. T. Moh in [10]. This 
statement is also true for free groups; we refer to [227]. For free associa- 
tive algebras of rank greater than 2, this statement does not hold; see 
Theorem 9.3.1. It does hold for the free associative algebra of rank 2; see 
Proposition 8.3.3. 

It follows from Theorem 12.5.15 that if u and v are elements of A, 
and A/(u) and A/(v) are isomorphic free algebras, then there is an au- 
tomorphism of A such that (p(u ) = v (actually, u and v are primitive 
elements). It turns out that there is no generalization of this result for 
arbitrary elements u and v. 

Theorem 12.5.16 ([254]). Let K be a field , L = L{x, y, z) the free Lie 
algebra over K with the set {x,y,z} of free generators. Let 

u = x- [[x,y\, [y,z}\, v = x + [[[y,z],x),y), 
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and let ( u ) and (v) be the ideals of L generated by the elements u and v, 
respectively. Then L/(u ) = L/(v), but there is no automorphism ip of L 
such that <p(u ) = v. 

Proposition 12.5.17 ([254]). Let K be a field, L = L(x,y) the free Lie 
algebra over K with the set {x,y} of free generators. Let 

U = x- [[[x, y\,y\, [x, y]}, v = x - [[[x, y], x], y\, 

and let ( u ) and (v) be the ideals of L generated by the elements u and v, 
respectively. Then L/(u ) = L/(v), but there is no automorphism ip of L 
such that p>(u) = v. 

Corollary 12.5.18. There are isomorphic one-relator Lie algebras L/(u) 
and L/(v) such that the isomorphism is not induced by any automor- 
phism of the ambient free Lie algebra L. Or, equivalently, there is no 
automorphism of L that takes the ideal (u) to (v). 

Problem 12.5.19. Does the statement of Theorem 12.5.15 take place for 
free Lie p-algebras and free color Lie p-superalgebras? 

In [14] S. A. Agalakov showed that a Lie algebra with one defining 
relation need not be residually finite. 



12.6 Primitive Elements and Endomorphisms 

In this section, we consider automorphic orbits of primitive elements of the 
free algebra A under the action of endomorphisms. 

For any element a of A, by a 0 we denote the leading /i- homogeneous com- 
ponent of a, where (l is a generalized degree function. Recall that a system 
of elements a\, . . . , is said to be reduced if for any i = 1 , . . . , k the 
component a° does not belong to the subalgebra generated by a? with 

j ± i ■ 

Proposition 12.6.1. Let ip be an endomorphism of A sending primitive 
elements to primitive elements, ai = ip{xi), 1 < i < n, SDT ^ DJI 2 for n = 2. 
Suppose that the system a \, . . . , a n _ 1 is reduced. Then: 

(i) There is a (i-homogeneous element h of H with rank (h) = n — 1, 
^(/i(a!j\...,a°-i)) > K a n)- 

(ii) rank^.-.-X-i) < n - 

(iii) There is a (i-homogeneous automorphism ^ of A such that elements 

. . . , d° n °t depend on some variable Xj of X. 

Proof. Let h = x\ r X2 . . . r a;n _ 1 rj l , where s • p(xi) > p(a^). It is clear that 
h = 0 if and only if n = 2 and DJI = SDT 2 . It is easy to see that the element h 
satisfies the condition (i). 
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(ii). The element / = x n + h(x i,...,x n _i) is primitive. Then by the 
conditions of the lemma, the element g = (p(f) is also primitive. By 
Theorem 12.5.1, there are elements ra*, 1 < i < n, of U(A ) such that 
ELi = 1 - Since 9 = a n + h{ai , . . . ,a n _i), 



E 



m* 



0&n 

dxi 




dh t 

(tti, . . . , CLn—1 ) — 1- 



Considering the component of the degree 



r + //(/i(aj, . . . , a®^)), where r = max{//(rai) — //(x*)}, 



we get 




( 12 . 8 ) 



where m' is the component of rrii of the degree r+fi(xi). By Theorem 11.1.1, 
since the system ai, . . . , a n _i is reduced, it forms a set of free generators 
of the subalgebra S of A that it generates. It follows from rank(h) = n — 1 
that the elements ^(af, . . . ,a°_ 1 ), 1 < j < n — 1, are left independent 
over U(S). Since U(A) is a free left [/(S')-module, it follows from (12.8) 
that 



E 

i — 1 



.da 0 , 

”‘te =0 ' 1£ -’ Sn - 



By Theorem 12.4.8, rank(uj, . . . , a°_ x ) < n. 

(iii). Fix teQ such that 

t > ma x{/ji(xi), | 1 < i < n, 1 < j < n—}. 

Let Y = {yi,...,y n -i}, = 2 t-y,(a°), 1 <i < n — 1, B = Fgn(Xur). 

Consider the homogeneous element 



b — CLiVl + 2/2 H + a n-l2/n-l £ B. 

As in the proof of Theorem 12.4.8, elements . . . , are left depen- 
dent over U(B). Since these elements are homogeneous, there is j with 
G By Proposition 12.3.2, there is a //-homogeneous au- 

tomorphism x/j of B such that = //i, 1 < i < n — 1, and an element 
^(6) does not depend on x^. Since //(x*) < t < //(//*) for any i, j, the el- 
ement ^(xi) does not depend on yj. Thus xf>(xi) G A, and the restriction 
of xp on A is a desirable automorphism. ■ 



Theorem 12.6.2 ([259]). Let (p be an endomorphism of A such that for 
any primitive element a of A the element <p(a) is primitive. Then (p is an 
automorphism of A. 
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Proof. We may suppose that ai = <p(xi) = Xi + 1 < i < n, where bi are 

linear combinations of monomials of degree not less than 2. If n = 2 and 
2Jt = 9tt 2 , then primitive elements of A have the form ax\ + /fo 2 , where 
a,/3 G F, (a, /?) ^ (0, 0). In this case, ^(xi) = Xi and p is an automorphism. 

Suppose that if n = 2, then 9# ^ 9H 2 . Consider the set of functionals 

{/x | there is an automorphism 

such that • • • Vv( a ™-i) are /x-bounded}. 

Let /x be a minimal element of this set with respect to the ordering -< . Con- 
sidering the composition if; o p, we may suppose that elements ai, . . . , a n _i 
are /x-bounded. We show that there is j such that ai,...,a n _i do not 
depend on Xj. Assume the contrary. As in the proof of Lemma 12.4.4, 
we find a functional /x o such that /xo ^ /x and the components 
/°i!.o( a i)>--->pio( a n-i) depend on every x € X. Using point (iii) of 
Proposition 12.6.1 and the proof of Lemma 12.4.5, we construct a func- 
tional /x i and an automorphism ^ of A such that /xi -< /x o and elements 
^(ai), . . . , ^(a n _i) are /xi-bounded. We have come to a contradiction with 
the choice of /x. Hence ai, . . . , a n _i do not depend on Xj. We may assume 
that j = n. Then ai, . . . , a n _i G if, and the restriction of p' on if is an 
endomorphism of H preserving the primitivity of elements. Using the induc- 
tion on n, we may assume that ip' is an automorphism of if, p'(xi) = x^ 
with i < n, and p(x n ) belongs to the ideal of A generates by x n . Since 
the element b = p(x n ) is primitive, A/(b) = if = A/(x n ), where (6) and 
(x n ) are ideals of A generated by b and x n , respectively. It follows that 
(' b ) = (x n ). By Theorem 12.5.13, p(x n ) = ax n , 0 ^ a G K. Thus, p is an 
automorphism of A. ■ 

Now we describe the situation in free Lie algebras and superalgebras. Let 
Lx = L(X) (the free color Lie superalgebra) in the case where char AT = 0 
and Lx = L P (X) (the free color Lie p-superalgebra) in the case where 
char K = p > 2. 

Theorem 12.6.3 ([269, 271]). Let X = (xi, . . . , x n }, and let p be an 
endomorphism of Lx such that for any primitive element u of Lx the 
element p(u) is also primitive. Then p is an automorphism of Lx- 

Problem 12.6.4. Let p be an endomorphism of the free Lie algebra L(X) 
over a field of positive characteristic such that for any primitive element h 
the image p(h) is a primitive element. Is it true that p is an automorphism? 

Problem 12.6.5. Let p be either an endomorphism of K[X] (or of K(X)) 
or of the free group F(X) such that for any primitive element h the image 
p(h) is a primitive element. Is it true that p is an automorphism? 

This problem has a close connection with the Jacobian conjecture for 
free algebras and groups and was discussed for free groups by V. Sh- 
pilrain in [352]. He pointed out that this problem is open (only in the 
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case n = 2 was it solved in the positive way, by V. Shpilrain [356] and 
S. V. Ivanov [154]). Under some additional assumptions, Problem 12.6.5 
for free groups was solved by S. V. Ivanov in [154]. D. Lee [205] settled the 
problem for an arbitrary rank n. A. van den Essen and V. Shpilrain in [124] 
obtained an analogous result for a primitive element h of the polynomial 
algebra in two variables (and of the free associative algebra of rank 2). 
Z. Jelonek [161] proved the statement of Theorem 12.6.2 for polynomial 
algebras over the field of complex numbers. 

Now we consider the images of linear combinations of free generators 
under an endomorphism and propose some problems. In Theorem 12.6.3, 
it is impossible to consider only images of such nonzero G-homogeneous 
linear combinations, as the following example shows. Let X = {x,y,z}, 
G = Z 2 , d(x) = 0, d(y) = d(z) = 1, L = L(X) the free Lie superalgebra, 
and p the endomorphism of L given by 

<p{x) = x + [y,z\ + [x,z,z\, ip{y) = y, <p(z) = z. 

Then it is clear that the image under p of any Z 2 -homogeneous nonzero 
linear combinations of x, y, and z is a primitive element, and at the same 
time p is not an automorphism of L. 

Moreover, even in the case of free Lie algebras, it is impossible in The- 
orem 12.6.3 to consider only images of linear combinations of elements of 
the free generating set. Indeed, the following theorem takes place. 

Theorem 12.6.6 ([271]). Suppose that n > 3, X = {x \, . . . ,x n }, the 
field K is not algebraically closed, and L = L(X) ( L — L P (X)) is the free 
Lie (p-) algebra . Then there exists an endomorphism p of L such that p is 
not an automorphism of L and at the same time for any nonzero linear 
combination a of elements of X the image p(a) is a primitive element. 

The situation in free associative algebras and in polynomial algebras is 
the same, as the following theorem shows. 

Theorem 12.6.7 ([268]). Suppose that K is a not algebraically closed 
field, n > 3, X = {x \, . . . ,x n }, and K[X] is the polynomial algebra 
over K. Then there exists an endomorphism p of K[X ] such that p is 
not an automorphism of K[X\ and at the same time for any nonzero linear 
combination a of elements of X the image p{a) is a coordinate polynomial. 

Theorem 12.6.8 ([268]). Suppose that K is a not algebraically closed 
field, n > 3, X = { x \ , . . . ,x n }, and K(X) is the free associative algebra 
over K. Then there exists an endomorphism p of K(X) such that p is not 
an automorphism of K(X) and at the same time for any nonzero linear 
combination a of elements of X the image p(a) is a primitive element. 

But for algebras of rank 2 over a field of zero characteristic, the situation 
is different. For two- variable polynomial algebras [68] (for free associa- 
tive algebras of rank 2 [255]), it is true that every endomorphism that 
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takes nonzero linear combinations of generators to primitive elements is an 
automorphism. 

Problem 12.6.9. Let K be an algebraically closed field and p an endo- 
morphism of a free Lie algebra L of finite rank such that for any nonzero 
linear combination a of elements of X the image p(a) is a primitive element. 
Is it true that p is an automorphism of L? 

Problem 12.6.10. Let K be an algebraically closed field and p either an 
endomorphism of the polynomial algebra K[X ] or of the free associative 
algebra K(X) such that for any nontrivial linear combination h of elements 
of X the image p(h) is a coordinate polynomial of K [X] (or a primitive 
element of K(X)). Is it true that p is an automorphism? 



12.7 Inverse Images of Primitive Elements 

Let if be a field and X = {xi , . . . , x n } a nonempty finite set. In this section, 
we study combinatorial properties of endomorphisms of free algebras F(X) 
by considering inverse images of primitive elements. 

In this section, F = F(X) is a free if-algebra (without a unit element) on 
a finite set X of free generators in one of the following varieties of algebras 
over a field K : the variety of all algebras, the variety of Lie algebras, the 
variety of Lie p-algebras, varieties of color Lie superalgebras, varieties of 
color Lie p-superalgebras, or varieties of all nonassociative commutative 
and anticommutative algebras. We start with inverse images of primitive 
elements under injective endomorphisms. 

Theorem 12.7.1 ([255]). Let u be an element of a free algebra F = F(X), 
v a primitive element of F , and p an injective endomorphism of F such 
that p{u) = v. Then u itself is a primitive element of F. 

Proof. Since v is a primitive element of F, there is a set Z = {z i, . . . , z n } 
of free generators of the algebra F such that v = z \ . 

Let fi = p(zi ), 1 < i < n. By a finite number of elementary transfor- 
mations, we transform the set {/i, . . . , f n } to a reduced (relative to Z) set 
{</i, • • • , g n } (since p is injective, no zeros appear at any step of this proce- 
dure). After applying the same sequence of elementary transformations to 
the set Z of free generators, we get a set Y — {yi, . . . , y n } of free generators 
of the algebra F such that p(yi) = #*, 1 <i <n. 

Now we write the element u as u = u(yi, . . . , y n ). We have 

p(u) = u(gi , . . . ,ffn) = Zi- (12.9) 

If £z(u) 1, then £z(p(u^ 1, and we arrive at a contradiction 

with (12.9). Therefore, u is a linear combination of the free generators 
zi , . . . , z n . It follows from (12.9) that this linear combination is nonzero. 
Thus u is a primitive element of the free algebra F. ■ 
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A statement of Theorem 12.7.1 for polynomial algebras in two variables 
over a field of zero characteristic was proved by L. A. Campbell and J.- 
T. Yu [65]. For free associative algebras of ranks 2 and 3, it is an open 
problem. For rank greater than 3, the answer is negative due to V. Shpilrain 
and J.-T. Yu [367]. 

Now we consider a more general situation. 

Theorem 12.7.2 ([255]). Let u \ , . . . , Uk be elements of a free algebra 
F = F(X) of rank n , {z \, . . . , Zk} a primitive system of F, 1 < k < n, and 
let (p be an injective endomorphism of the algebra F such that p(ui) = Z{ , 
1 < i < k. Then {u\, . . . ,Uk] is a primitive system of the algebra F. 
Furthermore, if k = n, then p is an automorphism of the algebra F. 

Proof. Complete the system { z \ , to a free generating set Z of the 

algebra F by some elements {zfc+i, . . . , z n }. The same argument as in the 
proof of Theorem 12.7.1 shows that there is a set Y = {yi , . . . , y n } of free 
generators of the algebra F such that 

{ffi, • • • , 9n I 9i = <p{yi), 1 <i<n} 

is a reduced subset of F relative to the free generating set Z. We write the 
elements u\,. . . ,Uk in the alphabet Y: Ui = Ui(yi , . . . , y n ), 1 < i < k. We 
know that 

<p(ui) = Ui(gi, . . . , g n ) = Zi, 1 <i<k. (12.10) 



Therefore 

n 

Ui = ^2 a ijVji where E K, 1 < i < k, 

3 = 1 

and if ^ 0 for some i,j , then — 1 (otherwise, we get a con- 

tradiction with (12.10) since the terms of degree > 1 cannot cancel 
out). 

It follows from (12.10) that the elements . . . , Uk are linearly indepen- 
dent. Thus, {u \, . . . , Uk} is a primitive system of elements of the algebra F, 
which consists of linear combinations of the free generators y\, . . . ,y n . If 
k = n, then U = {ui , . . . , u n } is a basis of the vector space generated by the 
elements y \, . . . , y n . Hence U is a free generating set of the algebra F, and 
the endomorphism p given by p(ui) = Zi, 1 < i < n, is an automorphism 
of the algebra F. ■ 

Now we consider inverse images of primitive systems under arbitrary 
endomorphisms. Recall that a subalgebra H of F is a retract if there exists 
an ideal I of F such that F = H 0 1. 

Theorem 12.7.3 ([255]). Let U = {ui , . . . , ui} be a subset of a free alge- 
bra F = F(X) of rank n > l, {zi , . . . , z{} a primitive system of F, H the 
subalgebra of F generated by U, and p an endomorphism of the algebra F 
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such that <p(ui ) = zi, 1 < i < Z. Then H is a retract of the algebra F. 
In particular, if l — n, then ip is an automorphism, and U is a set of free 
generators of the algebra F. 

Proof. In view of Theorem 12.7.2, we may assume that ip is not injective 
(obviously, a primitive system generates a retract). 

Complete the set {z \, . . . , z{\ to a set Z of free generators of the alge- 
bra F by some elements {zj+i, . . . , z n }, and let fa = <p(zi), 1 < i < n. 
By a finite number of elementary transformations, one can transform the 
set {/i, . . . , f n } into a set {<?i, . . . ,g n } such that for some k, 1 < k < n, 
the set {<7i, . . . is reduced relative to the alphabet Z, and gi — 0 for 
i > k (since ip is not injective, we have k < n). This sequence of elementary 
transformations gives rise to an automorphism ^ of the algebra F such 
that ^(xi) = y u ip(yi) = g u 1 < i < n. 

Now we write the elements u\,. . . ,ui in the following form: 

Ui{yi,...,y n ) = u^ 1) (y 1 ,...,y k ) + u\ 2 \ 1 <i<l, 

where for each i = 1, . . . , l the element up depends only on the free gener- 

( 2 ) 

ators 2/1, . . . , yfc, and the element u\ } belongs to the ideal of the algebra F 
generated by the free generators yk+ 1, . . . , y n - We know that 

tp(ui) = Ui(gi, ...,g n ) = uPigi, ...,g k ) = z u l<i<l. 

It follows that 

k 

U f ] = 

j = 1 

where iz(9j) — 1 whenever ^ 0. Hence Z < k. 

Let ip be a linear automorphism of the algebra F such that ^(yi) = 
for 1 < i < Z, and 'iptyj) = yj for Z < j < n. Denote y\ — 'ipiyi), 1 < i < n. 
Let J be an ideal of the algebra F generated by the free generators y' with 
Z < j < n. It is clear that Ui = y[ (mod J), 1 < i < Z. Thus, F = H 0 J, 
whence H is a retract of the algebra F. ■ 

For a description of retracts of free algebras F, see Section 13.2 in 
Chapter 13. 
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Let if be a field, char if / 2, and let X be a finite set, X = {#i, . . . , x n }. 
In what follows, F = F(X) denotes the free if-algebra without the unity 
element on the set X of free generators of one of the following varieties 
of algebras over a field if: the variety of all algebras, the variety of Lie 
algebras, varieties of color Lie superalgebras, the variety of Lie p- algebras, 
varieties of color Lie p-superalgebras, and varieties of commutative and 
anticommutative algebras. 

Fixed points of endomorphisms and automorphisms of algebraic systems 
are under intensive consideration by many algebraists. An element u is 
called a test element if for any endomorphism </? it follows from p{u) = u 
that ip is an automorphism. This definition was explicitly given by V. Sh- 
pilrain in [352]. But the history of test elements goes back to J. Nielsen 
and W. Dicks. A classical result of Nielsen [293] states that an endomor- 
phism x /; y — >► g of the free group F<i with two generators x,y is an 
automorphism if and only if [/,#] is conjugate to [x,y\. Hence the commu- 
tator [x,y\ = xyx~ l y~ l is a test element of i^. W. Dicks [93, 94] proved 
a similar result for the free associative algebra K(x,y) of rank 2. In articles 
[110, 124, 128], test elements of polynomial algebras and of free associative 
algebras are under consideration. 

Test elements of free groups have been of interest for many years. Exam- 
ples of such elements were given in [82, 100, 115, 131, 132, 147, 151, 165, 
293, 314, 316, 317, 318, 319, 320, 321, 352, 355, 383, 417, 418]. In [383], 
E. C. Turner proved that test elements for monomorphisms of free groups 
of finite rank are exactly the elements with maximal rank (an element of 
the free group has maximal rank if it does not belong to any free factor 
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of the group) and that test elements for endomorphisms of free groups are 
elements not contained in proper retracts; see our Section 2.2. The analogs 
of these results for free Lie algebras and free color Lie superalgebras and 
p-super algebras were proved by A. A. Mikhalev and A. A. Zolotykh in 
[277] and by A. A. Mikhalev and J.-T. Yu in [263], respectively. For free 
((anti)commutative) nonassociative algebras, these results were obtained 
by A. A. Mikhalev, U. U. Umirbaev, and J.-T. Yu in [259]. 

A subalgebra H of F is called a retract if there is a homomorphism 
p: F — >> H such that p is the identity mapping on H (p is the retraction 
homomorphism). Obviously, a subalgebra H of F is a retract if and only if 
there exists an ideal I of F such that F = H 0 /. 

Let F n be the free group of rank n, h a nonidentity element of F n , and 
Orb(h) the orbit of h under the action of the automorphism group of F n . 
The following problem was raised by V. Shpilrain. 

Problem 13.0.1 ([352]). Suppose that (p is an endomorphism of F n such 
that 



<p(Orb(h)) C Orb(h) 

for some nonidentity element h of F n . Is then ip an automorphism of F n l 

For n = 2, the positive solution was given by S. V. Ivanov in [154] 
and by V. Shpilrain in [356]. For an arbitrary n, the problem was set- 
tled by D. Lee [206]; see also Theorem 4.3.1. It is interesting to consider 
the same problem for free algebras (Lie, associative, relatively free alge- 
bras). V. Drensky and J.-T. Yu in [109] solved this problem for n = 2 
for free Lie algebras and for free metabelian associative and Lie algebras. 
One may consider a special case when h is a primitive element; i.e., h be- 
longs to a free generating set of the free group (or of the free algebra). In 
[269, 271], A. A. Mikhalev and A. A. Zolotykh solved the problem for free 
Lie algebras and for free color Lie (p-) super algebras assuming that h is 
a primitive element. A. van den Essen and V. Shpilrain in [124] obtained 
an analogous result for a primitive element h of the polynomial algebra in 
two variables (of the free associative algebra of rank 2), and they proved 
that if the Jacobian conjecture (that is, that any endomorphism of the 
polynomial algebra over a field of characteristic zero with the invertible 
Jacobian matrix is an automorphism) is true, then the problem has an 
affirmative answer for a primitive element of the finitely generated polyno- 
mial algebra over an algebraically closed field of characteristic zero. Finally, 
Z. Jelonek [161] proved this statement for polynomial algebras over the field 
of complex numbers (his proof used methods of algebraic geometry). In 
[268, 271] A. A. Mikhalev, J.-T. Yu, and A. A. Zolotykh showed that if K 
is a nonalgebraically closed field, then it is not enough to consider only im- 
ages of nonsingular linear combinations of free generators. S. V. Ivanov in 
[154] solved Problem 13.0.1 (under some additional conditions) for a prim- 
itive element h of F n . We have exposed exact solutions of Problem 13.0.1 
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for primitive elements of free algebras of Schreier varieties of algebras in 
Section 12.6 of Chapter 12. In the general situation, positive solutions of 
the problem were obtained by A. A. Mikhalev and J.-T. Yu [264] and by 
A. A. Mikhalev, U. U. Umirbaev, and J.-T. Yu [259]. 

In Section 13.1, we consider test elements of the algebra F = F(X): 
we prove that test elements for monomorphisms of the algebra F are the 
elements of the maximal rank (Theorem 13.1.1); test elements for endo- 
morphisms of F are elements that do not belong to any proper retract 
of F (Theorem 13.1.4). Section 13.2 characterizes retracts of the algebra F 
(Theorem 13.2.1). In Section 13.3, we solve Problem 13.0.1 for finitely gen- 
erated free algebras of Schreier varieties of algebras: if an endomorphism p 
of the algebra F preserves the automorphic orbit of a nonzero element of F, 
then actually p is an automorphism (Theorems 13.3.1-13.3.3). 

After that, we study generalized primitive elements of free algebras and 
their automorphic orbits. 

Let il be a variety of groups defined by the set of laws 53, F the free 
group, and let 53(F) be the verbal subgroup defined by 53. An element g 
of F is said to be il-generic if g e 53(F) but g £ 53(G) for every proper 
subgroup G of F. 

In the articles [82, 100, 131, 132, 225, 319, 352, 374, 375, 417] generic 
elements of free groups were studied. In particular, series of generic elements 
of free groups were constructed. In [82] it was shown that for a variety with 
a solvable word problem there is an algorithm to recognize generic elements. 

An almost primitive element (APE) is an element of a free group F that is 
not primitive in F but is primitive in any proper subgroup of F containing 
it. APEs of a finitely generated free group are test elements [82]. But there 
are test elements of free groups that are not generic: the element x 2 [y 2 , z] 
is a test word but is not generic [82]. An element x 2 yx~ l y ~ 1 is an APE of 
the free group F(x, y ), but it is not a test element. In general, generic does 
not imply APE [132]. At the same time, in [132] it was proved that if il is 
a nontrivial variety defined by the set of laws 53 and w 6 53(F), and w is 
an APE of F, then w is il-generic and hence a test element in F. A series 
of interesting results were obtained for almost primitive elements of free 
groups in [60, 131, 132, 182, 183, 318, 319, 320, 321]. See Section 3.1 for 
further discussion. 

Section 13.4 deals with almost primitive and test elements of free algebras 
with the Nielsen-Schreier property. We show that almost primitive elements 
are elements of maximal rank; see Proposition 13.4.1. If the free algebra F 
is a free product of its subalgebras A and F, and elements a and b are 
homogeneous almost primitive elements in algebras A and F, respectively, 
then the element a + b is an almost primitive element in F; see Proposi- 
tion 13.4.6. This result gives us a possibility to construct in Theorem 13.4.7 
a series of almost primitive elements. We discuss also a connection between 
properties of elements to be almost primitive and to be test elements. 
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A vector of Laurent polynomials is called A-unimodular if its components 
generate the augmentation ideal of the whole algebra. V. A. Artamonov 
[20] proved that the general linear group acts transitively on the set of 
all A-primitive vectors (for unimodular vectors, this result was proved by 
A. A. Suslin [378]). V. Shpilrain [356] considered A-primitive elements of 
the free group. He proved that in a free group F 2m any A-primitive ele- 
ment is an automorphic image of [x\, x 2 ][x3, £4 ] . . . [a; 2m _i, x 2m ], there are 
no primitive elements of the group F n if n is odd, and in a free group F of 
finite rank, the group Aut(F) acts transitively on the set of all A-primitive 
elements; see Section 3.1. By A we denote the left ideal of the free associa- 
tive algebra K (X) generated by the set X. Following [356], we say that an 
element u of L(X) is a A-primitive element if the elements . . . , 
generate A as the left ideal of K(X). 

In Section 13.5, we prove Theorem 13.5.1: if the rank n of the free Lie 
algebra L is even, n = 2m, then any A-primitive element of L is an au- 
tomorphic image of [xi,x 2 ] H b [x 2m _i,x 2m ], there are no A-primitive 

elements of L if n is odd, and the group of automorphisms of the algebra L 
acts transitively on the set of all A-primitive elements. 

In Section 13.6, we consider generic elements of free Lie algebras. Let 11 be 
a variety of Lie algebras over a field K defined by the set of identities 2J, L = 
L(X) the free Lie algebra with the set X of free generators, and let 5 3(L) be 
the verbal ideal of L defined by 2J. An element / of L is said to be il-generic 
if / G 9J(L) but / ^ fO{H) for every proper subalgebra H of L. Let 21 be 
the variety of abelian Lie algebras. An element u of the free Lie algebra L is 
2l-generic if u G [L, L], and if u G [H, H] for some subalgebra H of L, then 
H = L. As in the situation with free groups (see [82]), since free Lie algebras 
of finite rank are Hopfian, generic elements are test elements. Propositions 
13.6.1 and 13.6.3 give new examples of almost primitive elements of free Lie 
algebras. In Proposition 13.6.4 and Theorem 13.6.5, we construct series of 
examples of 2l-generic elements of free Lie algebras. Theorem 13.6.7 gives 
an example of an 2l-generic element of the free Lie algebra L(x,y ) that is 
not an APE of this algebra. In Theorem 13.6.8, we show that the element 
u = [xi,...,x n ] is an 9l n -generic element of the free Lie algebra L(X), 
where 9t n is the variety of nilpotent Lie algebras of degree not less than n. 

In this chapter, we propose a series of interesting open problems for 
future research, Problems 10.3.2, 13.1.5, 13.2.2, 13.3.4, 13.4.8, and 13.6.9. 



13.1 Test Elements 



In this section, we characterize test elements for monomorphisms (injective 
endomorphisms) and for endomorphisms of free algebras F(X) of finite 
rank. 
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Theorem 13.1.1 ([264]). Let K be a field , X = {#i, . . . ,x n }, u G F = 
F(X), rank(u) = n, and ip a monomorphism of F such that (f(u) = u. 
Then ip is an automorphism of F. 

Proof. Let u = u° x +u* x , where u° x is the leading component of u relative 
to the degree in the alphabet X. Let also u x = u x +u x , u x = u x °+u x *, 
and so on. 

For any set Y of free generators of the algebra F, we consider the infinite 
row 



Y(u) = (£y(^),rank(^y),^y(iiy),rank(iiy), . . . ). 

We compare such rows lexicographically starting from the left side. 

Changing if it is necessary the set X of free generators, one can assume 
that X has the following minimal property: 

x (u) = min Y (u), 

where Ct = {Y \ Y is a set of free generators of F}. 

Let hi = <p(xi ), i = 1, . . . , n, and let H = {hi, . . . , h n } and B be the 
subalgebra of F generated by H. Since ip is a monomorphism of F, H is an 
independent subset of F. There exists a sequence of elementary transfor- 
mations R such that R(H) = V = {ui, . . . , u n }, V{ ^ 0 for all i = 1 , . . . , n, 

V is a reduced subset, and V generates the subalgebra B. 

Let Y — {yi, . . . , y n } — R(X). It is clear that Y is a set of free generators 
of F, Vi = ip(yi) for 1 < i < n. 

Let u be the presentation of u in generators from Y. Then, using the 
assumptions of the theorem, we get 

u(x i,..., x n ) = S(yi,...,2/ n ) = u(v i,...,u n ), (13.1) 

where u(v 1 , . . . , u n ) is obtained from u(y \, . . . , y n ) by changing yi on 
i = 1, . . . , n. 

On the contrary, assume that (p is not an automorphism of F. Since 

V is a reduced set, it means that lx(vj) > 1 for some j. Without loss 
of generality, we may suppose that v\ = xi,..., v r = x r , r < n, and 
£x{vj) > 1 for all j = r + 1, . . . , n (possibly r = 0 and £xiVi) > 1 for all i). 

It follows from (13.1) that u(x 1 , . . . ,x n ) 6 B. Consider now the leading 
part of the element u : 

u(x 1 , . . . , x n ) = u° x + u x , 

Ux — i^x (^1 j • • • ? x r ) + u x , 

where u x is the component of u° x that does not depend on generators 

2V+ 1? • • • j ^n* 

If u° x = 0, then we rewrite u as an element of the subalgebra B with the 
set V of free generators. Since V is a reduced subset, 

t V {u(x s n )) < £x(u(x i,...,x n )). 
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Taking into account (13.1), we get 

Iy . . . , 2 /n)) Ixi^ip^li • • • 5 *^n))* 

This gives a contradiction with the minimal property of X relative to u. 

In the case where u° x ^0 we have 

u(x i,...,X n ) =u{v = 

where 

Uy = «y (t>l, • • • , V r ) + Uy{v i, . . . ,U n ), 

and fty 1 is the component of Uy that does not depend on iv+i, . . . , v n . 

It is evident that 

U^(v!,...,V r ) =U°x{xi,...,X r ). 

If ^ 0, then bearing in mind that V is a reduced subset, we obtain 

tx(u(vu...,v r )) > l x (u). 

It gives a contradiction with (13.1). 

Therefore 

Uy = u°x{x 1 ,...,x r ) = u°x(v i,...,V r ). (13.2) 

Now we consider the component u ° x . If u° x 7 ^ 0, then it follows 
from (13.2) that 

rank(-Uy) = ranks(uy) = rank(^) < rank(u(xi, . . . ,x n )). 

As above, it gives a contradiction with the minimal property of X. Hence 
u° x does not depend on x r +i, . . . , x n . Since v\ = xi,...,v r = x r (i.e., 
u° x = Uy), by analogy we see that the elements u x ,u x °, ... do not depend 
on , . . . , Xji . 

Thus, the element u does not depend on x r +i, . . . , x n , and we have come 
to a contradiction with the condition rank(?i) = n. ■ 

As a corollary, we have the following theorem: 

Theorem 13.1.2 ([264]). Let X = {#i, . . . , x n }, u G F(X), rank(u) = n, 
and let ip be a monomorphism of the free algebra F = F(X ). Then ip is an 
automorphism of F if and only if the element p(u) belongs to the orbit ofu 
under the action of the automorphism group of F (p(u) G Orb(u)). 

Proof. If ip is an automorphism of F, then our statement is obvious. Now 
let p(u) G Orb('u). Then there exists an automorphism if of F such that 
tp(u) = if( u ). For the monomorphism p = ipif~ l of F and for the element 
v = if(u), we have 

p(v) = pip -1 (if(u)) — p(u) = if(u) = v, 

and rank(v) = n. Applying Theorem 13.1.1, we get that pif -1 is an auto- 
morphism. Thus, ip is also an automorphism of F. ■ 
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Lemma 13.1.3. Let K be a field , char AT ^ 2, \X\ > 2 , and let u be 
a homogeneous element ofF(X) of maximal rank. Then u is a test element. 

Proof. Let p be an endomorphism of F(X), hi = p(xi), 1 < i < n. Since 
\X\ > 2 and u is a homogeneous element of maximal rank, £(u) > 2. It 
follows now from p(u) = u that elements hi, 1 < i < n, are nonzero, 
and their component of minimal degree hi has the length 1. Since u is an 
element of maximal rank, elements hi, 1 < i < n, are linearly independent. 
Hence p is a monomorphism of F(X). Since u is an element of maximal 
rank, it follows from Theorem 13.1.1 that p is an automorphism of F(X). 



Note that the statement of Lemma 13.1.3 for free Lie algebras 
was proved by V. Shpilrain in [354]. Theorems 13.1.1 and 13.1.2 for 
free color Lie (p-)superalgebras were obtained by A. A. Mikhalev and 
A. A. Zolotykh [277]. 

The statement of Theorem 13.1.1 is not true in general for elements u 
with rank(u) < n. Indeed, if u does not depend on x\, then one can consider 
the endomorphism p given by p(x i) = x\ • #2 and p{xf) = Xi for any 
i > 1. At the same time, it is impossible to omit the condition that p is 
a monomorphism. Indeed, let X = {x\,X 2 }, u = x\ + x\ • X 2 , p{x i) = 
u , and p(x 2 ) = 0. Then rank(u) = 2 = \X\, p(u) = u , but p is not 
an automorphism of F(X). The reason for this fact is that if U is the 
subalgebra of F(X) generated by the element u and J the ideal of F(X) 
generated by X 2 , then F(X) = U 0 J; that is, the element u belongs to the 
proper retract U of F(X). 

Theorem 13.1.4 ([263, 264]). Let X = {#i, . . . ,x n }. Test elements of 
the free algebra F=F(X) are precisely those elements not contained in any 
proper retract of F. 

Proof. Let p be a nontrivial endomorphism of the algebra F such that p 
is not an automorphism of F, 0 ^ it G F, and p(u) = u. 

If p is a monomorphism of F, then, by Theorem 13.1.1, rank(u) < n; i.e., 
the element u belongs to a proper free factor of F that is a proper retract 
of F. Suppose now that p is not a monomorphism. Let 

00 

1=0 

Since u ^ 0, p°°(F) ^ 0, we may suppose that there exists a free generating 
set Y of F such that for some s, 1 < s < n, we have 

<p(yi) =Ui t^O, 1 < i < s, 



and 



<p{yj) = 0 for s <j < n, 
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the subset {ui , . . . ,u s } is reduced modulo the ideal J of F generated by 
the free generators y s +i, . . . , y n . Let Ui = Ui + u*, 1 < i < s, where Ui is the 
component of Ui that depends only on y \ , . . . , y s , and u * G J. It is clear that 
Ui 7^ 0 for all i, 1 < I < s - In the same way, we write down the element u in 
the form u = u + u*. Again it is obvious that u = u(y \ , . . . , y s ) 7^ 0. Since 

u = p(u) = tp(u) + <p{u*) = p(u ), 



we get 



u° = (tp(u)) ; 

that is, 

u°{yi,...,y s ) = u°{u 1 ,...,u s ) i°). 

We may assume that the component u° depends on generators yi, . . . ,y t , 
where 1 < t < s. Then we get £(ui) = i{yi ), 1 < i < s. If t = 5, then the 
subalgebra H generated by ui, . . . , u s is a proper retract of F, and u G H. 
If t < 5, then we consider the decomposition of the component u into the 
sum of homogeneous components (relative to the length function) 

u = u° + u 00 + . . . , where u°° = (u*)°, 

and so on. Using the same arguments, we obtain that £(uj) = £{yj) for all j 
such that u depends on yj. One can suppose that the element u depends 
on yi, . . . , 2/ r , where t <r < s. Since <p(u*) = 0, 



u = u(yi,...,y r )+u* =ip(u) = u(m, . . . ,u r ). 

Therefore the element u belongs to the subalgebra H' generated by the 
elements ui, . . . , u r (H f C H C F). Using a nonsingular linear transforma- 
tion of free generators from y, we may assume that Uj = yj + u*, where 
1 < j < r, u* e J. Thus H' is a proper retract of F, u G H f . 

We have proved that every element u of F not contained in any proper 
retract of F is a test element for automorphisms of the algebra F. On 
the other hand, let H be a proper retract of F, 0 / u G If, and ip the 
corresponding retraction homomorphism. Then p is not an automorphism 
of F but p(u) = u. Namely, u is not a test element. This completes the 
proof. ■ 

Problem 13.1.5. Construct an algorithm to recognize test elements of 
free algebras of Schreier varieties of algebras. 

L. P. Comerford [82] proved that there is an algorithm to recognize test 
elements of free groups. 
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13.2 Retracts 

In this section, we show that retracts of free algebras F(X) of finite rank 
have the same description as retracts of free groups of finite rank (as for 
retracts of free groups, see, for example, the monograph [231]). For more in- 
formation about subalgebras and ideals of free algebras of Schreier varieties 
of algebras, we refer to the monographs [32, 33, 57, 275, 313]. 

Theorem 13.2.1 ([57, 264]). Let K be afield, X = {x\, . . . ,x n }, and H 
be a nonzero subalgebra of the free algebra F=F(X). Then H is a proper 
retract of F if and only if there exist a set Y = (t/i, . . . , y n } of free genera- 
tors of the algebra F, an integer r, 1 < r < n, and a set U = { u\ , . . . , u r } 
of free generators of the algebra H such that 

Ui = yi + u *, 1 < i < r, (13.3) 

where the elements u* belong to the ideal of the algebra F generated by the 
free generators y r + 1 , . . . , y n . 

Proof. If the subalgebra H has a set (13.3) of free generators, then we 
consider the homomorphism p: F — >• H given by p(yi) = U{ with 1 < 
i < r, and p(yj) = 0 with r < j < n. It is clear that p is a retraction 
homomorphism, F = H®I, where I is the ideal of the algebra F generated 
by the elements y r +i, • • • , Vn- Thus if is a nontrivial proper retract of F. 

Now let H be a nontrivial proper retract of F, p: F H the 
corresponding retraction homomorphism, and 

hi = p{xi), 1 < i < n. 

Applying a sequence of elementary transformations to the set {bi, . . .,b n }, 
we come to a set of free generators { u \ , . . . ,u r } of the subalgebra H. If 
r = n, then p is a monomorphism, and H is not a retract of F. Therefore 
r < n. Applying the same sequence of elementary transformations to the 
set X , we obtain the set 



Y = {yi,...,y„} 
of free generators of the algebra F such that 

p{y%) = Ui, 1 <i<r, 



and 



p{yj) = 0, r + 1 < j < n. 

Without loss of generality, one can assume that the set {u±, . . . ,u r } is 
reduced relative to the subset {t/i , . . . ,y r }- Hence Kerp is equal to the 
ideal generated by the subset 

{Vj I r + 1 < j < n}. 
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Since p is the identity mapping on if, 

p{ui) =Ui = p(yi) with 1 < i < r. 



Hence 



Ui = yi + u * with 1 < i < r, 

where the elements u*, 1 < i < r, belong to the ideal of the algebra F 
generated by the subset {y r + 1 , . . . , y n } of Y. This completes the proof. 



One can try to solve Problem 13.1.5 using Theorems 13.1.4 and 13.2.1. 
In general, the situation seems to be rather complicated. But for free two- 
generated algebras with the Nielsen-Schreier property, the situation for 
constructing an algorithm to determine test elements is much simpler. By 
the description of retracts of finitely generated algebras with the Nielsen- 
Schreier property given in Theorem 13.2.1, any proper retract of F(x,y) 
is generated by one element w. There is a free generating set {u,v} of 
F(x,y) such that w = u + /, where each monomial in / depends on v. 
Using Theorem 13.1.4, one may easily see that test elements of the free 
Lie algebra L = L(x,y) are nonzero elements of the commutator algebra 
[L, L], this fact was noted in [109]. For other classes of free algebras with the 
Nielsen-Schreier property, we have to construct an algorithm to determine 
generating elements of retracts and an algorithm to determine whether an 
element belongs to the subalgebra generated by a retract generated by one 
element. 

We illustrate these algorithms for a free Lie p-superalgebra L = L p (x,y) 
of rank 2. Any element w that generates a proper retract of L has a form 
u + /, where L = L(u, v) and each monomial of the element / depends 
on v. Therefore, if a given element a of L has no summands of the de- 
gree 1, then it is not an element generating a proper retract. If it has linear 
summands, then by using an automorphism we may write it in a form 
a = x' + g, where L = LP(x! , y') and all monomials of the element g have 
degree greater than 1. Now, to determine whether a generates a retract or 
not, it is enough to determine whether g in the standard presentation has 
the monomials that do not depend on y f (these monomials are pLpowers 
of x, t > 1). Now suppose that for a given element a we decided that a 
is not an element that generates a retract, but possibly it belongs to the 
subalgebra generated by an element 6, and b generates a proper retract. We 
suppose that K is a perfect field given constructively, with an algorithm 
of extracting p-roots. Then the leading monomial a 0/ of a is a p^-power of 
a leading monomial b of of b. We take the minimal possible such t and write 
b in the form b = ab°' + 5* , where a G K and b* is a linear combination 
with undetermined coefficients of monomials that are less than b°' (at first 
by the usual degree and then lexicographically). Since a belongs to the 
subalgebra generated by 5, [a, b] = 0. It gives us a system of linear equa- 
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tions on coefficients. By solving this system we complete the algorithm. 
A similar algorithm may be used for free Lie superalgebras and for free 
Lie p-superalgebras of rank 2. For free Lie super algebras, we may use the 
algorithm to extract square roots from even elements; this algorithm was 
described in monographs [33, 275]. For free nonassociative algebras and 
for free (anti)commutative nonassociative algebras, the situation is differ- 
ent, and we have no analog of the condition [a, b] = 0. Thus, to find the 
element 6, we have to solve some system of algebraic equations. 

Problem 13.2.2. Is it true that the intersection of two retracts of the free 
algebra F=F(X) is a retract of this algebra? 

For free groups, Problem 13.2.2 was solved in the affirmative by 
G. Bergman in [47]. 



13.3 Endomorphisms and Automorphic Orbits 

In this section, we prove that an endomorphism preserving the automorphic 
orbit of a nonzero element of F(X) is an automorphism of this algebra. 

Theorem 13.3.1 ([264]). Let K be a field, X = {x\, . . . ,x n }, u be 
a nonzero element of the algebra F = F(X), Orb('u) the automorphic orbit 
of the element u, and <p an endomorphism of F such that (p(Orb(u )) C 
Orb(u). Then p> is a monomorphism of F. 

Proof. It is clear that (p(u) = (i{u) for some automorphism /i of F. Then 
for the endomorphism if = we have ^(u) = u. If the element u does 

not belong to any proper retract of F, then by Theorem 13.1.4 the endo- 
morphism ^ is an automorphism of F, and therefore (p is an automorphism 
of F. 

Now let H be a proper retract of F, u 6 H. By Theorem 13.2.1, there 
is a set Y = {y ±, . . . , y n } of free generators of the algebra F = F(X) such 
that the subalgebra H is generated by the elements 

v i =zy i + hi , with 1 < i < Z, l < n, (13.4) 

where for all i the element hi belongs to the ideal of F generated by the 
elements j/j+i, . . . ,y n - 

Using a linear transformation of the set Y, one may suppose that the 
elements hi do not have linear summands. By h ^ we denote the minimal 
component (by the length in Y) of the element h. Changing the element h 
of Orb(u) and the set Y, one may suppose that the presentation (13.4) is 
such that the component h ^ depends on the minimal possible number of 
free generators from Y. We may assume that h^ depends on y\. If <p is 
not an automorphism of F, then there exists an automorphism ^ of F such 
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that (p(^(yi)) = 0 , and the nonzero elements of the set 

M Hvj)) I j > i} 

form a reduced subset relative to the alphabet Y. 

Consider the element v = ip(^(u)) and its minimal component Since 
— 0, either > ly{u^) and v £ Orb(u), or the compo- 

nent v W depends on a number of free generators fewer than 1. Bearing 
in mind that l is the minimal number in (13.4), we get v £ Orb(u). This 
contradiction completes the proof of the theorem. ■ 

Let if be a field, char K ^ 2, and X = {xi, . . . , x n } a G-graded set. 
By Lx we denote the free color Lie superalgebra L(X) in the case where 
char AT = 0 and the free color Lie p-superalgebra L P (X) in the case where 
char K = p > 2. 

Theorem 13.3.2 ([264]). Let K be afield, char if ^ 2, X = {xi, . . . , x n }, 
and let u be a nonzero element of the algebra L = Lx, Orb(u) the auto- 
morphic orbit of the element u, and ip an endomorphism of L such that 
<^(Orb(u)) C Orb(^). Then ip is an automorphism of L. 

Proof. By Theorem 13.3.1, ip is a monomorphism of the algebra L. If 
rank(u) = n, then by Theorem 13.1.1 99 is an automorphism of L. Suppose 
now that rank(u) = l < n and ip is not an automorphism of L. Then by 
Theorem 12.6.3 there exists a set Y = {yi, . . . , y n } of free generators of L 
such that ip(yi) is not a primitive element of L. 

Since rank(u) = /, there exists an automorphism ^ of L such that the 
element h = depends only on j/i, . . . , yi, h = h (yi, . . . , y{). Taking into 
account that <p(Orb(tz)) C Orb (u), we get that for the element 

ip(h) = hitpiyi), . . . ,<p(yi)) 
there is an automorphism 6 of L such that 

%>W) = Kyu---,vi)- 

Now we consider the following endomorphism £ of L: 

€{yi) = <p(y%) With 1 < i < l\ 

HVi) = 0 _ 1 ( 2 /?) with l <j <n. 

Let 

w = [... [[%i, • • • , yi ), yi+i], yi+ 2], • • • , y n \- 

Then it is clear that rank(u;) = n, and 

6{£(w)) = 6 >([. . . [[/i(y>(yi), . . . , <p(yi))], 0 _ 1 (ym)] • • • , 

= [• • • [h, yi+ 1 ] ...,y n ]=w. 

Since rank(u;) = n, by Theorem 13.1.1, 0£ is an automorphism of L. Hence 
£ itself is an automorphism of the algebra L. But the element £( 2 / 1 ) = ip(yi) 
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is not primitive. This contradiction completes the proof of the theorem. 



Let A be a finitely generated free algebra of the variety Mo (the variety 
of all algebras), 9Jti (the variety of all commutative algebras, char if ^ 2), 
or M 2 (the variety of all anticommutative algebras). 

Theorem 13.3.3 ([259]). Let u be a nonzero element of A , Orb (u) the 
automorphic orbit of the element u, and p an endomorphism of A such 
that <^(Orb(u)) C Orb(u). Then if is an automorphism of A. 

Proof. The proof can be given along the lines of the proof of Theo- 
rem 13.3.2 using Theorem 12.6.2. ■ 

Problem 13.3.4. Let u be a nonzero element of a free associative algebra 
K(X), Orb(u) the automorphic orbit of the element u, and ip an endomor- 
phism of K(X) such that <£>(Orb(u)) C Orb(u). Does it follow that ip is an 
automorphism of K{X)1 



13.4 Almost Primitive and Test Elements 

An almost primitive element (APE) of a free algebra F is defined as an 
element that is not primitive in F but is primitive in any proper subalge- 
bra of F containing it. We consider almost primitive elements of finitely 
generated free algebras of the main types of Schreier varieties of algebras. 

Proposition 13.4.1 ([265]). Almost primitive elements of a free alge- 
bra F are elements of the maximal rank ( that is, they do not belong to any 
proper free factor of F) . The reverse statement is not true. 

Proof. Let a be an APE of F and B a proper free factor of F. Then 
the element a is primitive in B , and thus a is a primitive element of F. 
If F — F(x,y ), a free algebra of rank 2, then the element ( xy)y has the 
maximal rank 2, but at the same time it is not primitive in the subalgebra 
generated by xy and y. Hence ( xy)y is not an APE of F. ■ 

Since elements of the maximal rank are test elements for injective endo- 
morphisms of F, it is interesting to realize the difference between almost 
primitive elements and test elements. 

Lemma 13.4.2. LetK be afield, char AT ^ 2, and letu be an APE ofF(X) 
whose component of degree 1 is equal to zero. Then u is a test element of 
F(X). 

Proof. By Theorem 13.1.4, test elements of the algebra F(X) are pre- 
cisely those elements not contained in any proper retract of F(X). Let H 
be a proper retract of F(X), u £ H. Since u is an APE of F(X), it is 
a primitive element of H. Since the automorphism group of H is generated 
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by elementary automorphisms, it follows from Theorem 13.2.1 that u has 
a nonzero linear component. This contradiction completes the proof. ■ 

Proposition 13.4.3 ([265]). Let K be a field, char K ^ 2. 

1. xy is an APE of F(x,y), and it is also a test element. 

2. x + xy is an APE of F(x,y), and at the same time it is not a test 
element; ( xy)z is not an APE of F(x,y, z), and at the same time it 
is an element of maximal rank. 

3. xx is an APE of the free nonass ociative algebra F(x), and it is also 
a test element; if y is an odd variable, then [; y , y] is an APE of the 
free Lie superalgebra L(y) of rank 1, and it is also a test element. 

4. x p is an APE of the free Lie p-algebra L p (x), and it is also a test 

2 

element; x p is a test element but not an APE. 

5. The element x—[[x, y\,y\ is not an APE of the free Lie algebra L(x , y); 
also it is not a test element, but it has the maximal rank 2; the element 
[[ x ,2/],2/] is not an APE, but it is a test element of L(x,y). 

6. [, x , y] + [[x, z\,z\ is an APE of the free Lie algebra L(x, y , z), and it is 
also a test element. 

7. If x and y are even, and z is odd, then [z,y] + [[z,x],x] is an APE 
of the free Lie superalgebra L(x , y\ z), and it is also a test element; if 
y and z are even and x is odd, then this element is neither APE nor 
a test element of L(y,z\x). 

8. //char K = p > 2, then the element x + [; y , z] + (adx) p (z) is an APE 
and it is not a test element of the free Lie algebra L(x,y,z), and it 
is not an APE of the free Lie p- superalgebra L p (x , y , z). 

Proof. Suppose that one of the elements of points 1-8 belongs to a finitely 
generated subalgebra H of F(X). We denote this element by u. Let x > 
y > z, and let {hi, . . . , h m } be a reduced set of free generators of H. By 
Theorem 11.1.1, the leading part u° of u is a polynomial in . . . , 

1. u = xy = u° . If u° is a linear combination of some of h°, 1 < i < m, 
then it is clear that u is a primitive element of H. Otherwise we may 
suppose that u° = h^h^, h\ = x, h^ = y . Then H = F{x,y), and H is not 
a proper subalgebra of F(x,y). By Lemma 13.4.2, xy is a test element of 
F{x,y). 

2. To show that u = x + xy is an APE of F(x,y), one may use the 
same arguments as in point 1 of the proof. Let (p be the homomorphism of 
F(x, y) given by p(x) = u and (p(y) = 0. Then p is not an automorphism, 
p(u) = u , and u is not a test element of F(x, y). 

For the element v = (xy)z, we set hi = xy , /12 = z. Let H be the 
subalgebra of F(x,y,z) generated by hi and / 12 . It is clear that {/ii,/i 2 } 
is a reduced set, and H is a proper subalgebra of F(x,y,z). At the same 
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time, v — hih 2 , and v is not a primitive element of H. It is obvious that v 
has the maximal rank. 

3, 4. To show that xx, x p , and [y, y] are APE and test elements of F(x), 
L p (x), and L(y), respectively, one may use arguments from point 1 of the 
proof. 

Let H be the subalgebra of L P (X) generated by the element w = x p , and 
2 

v = x p . Then if is a proper subalgebra of L p (x), v e H, v = u p , and v 
is not a primitive element of H. Thus x p is not an APE of L P (X). At the 
same time, the algebra L p (x) has no proper retracts. Hence any nonzero 
element of L p (x) is a test element. 

5. Let u = x - [[x,y],y], hi = [x,y] - x, h 2 = y, and let H be the 
subalgebra of L(x,y) generated by {hi,h 2 }. It is clear that {hi,h 2 } is 
a reduced set, H is a proper subalgebra of L(x, y), and u is not a primitive 
element of H (u = —(hi + [hi, h 2 ])). Therefore u is not an APE of L p (x , y). 
Taking the retraction homomorphism given by ip(x) = u , <p(y) = 0, we 
get <f(u) = u. Thus u is not a test element of L(x,y). At the same time, 
u is an element of the maximal rank in L(x, y) (otherwise it is a primitive 
element, which is not the case). 

It is clear that v = [[x,y],y] is not an APE of L(x,y), but v is a test 
element of L(x,y). 

6 . For u — [x,y\ + [[x,z],z], we have u° = [[x,z],z]. If u° is a linear 
combination of some /i°, then u is a primitive element of H. Otherwise we 
have to consider the following cases. In the first case h\ = [x,z], h% = z = 
h 2 . Then 

hi = [x, z] + ax + (3y + jz, a, /?, 7 € K , 

and u — [hi, h 2 \ = [x, y] — a[x, z] — (3[y, z]. If now h^ contains a summand 
[x,y], then u is a primitive element of H , and if h% = x, h\ = y , then 
{hi, h 2 ,hs, h 4 } is not a reduced set. 

In the second case, h\ = x = hi, h% = z = h 2 . Then 

u- [[hi,h 2 ],h 2 \ = [x,y\. 

If /13 has the summand [x,y\, then u is a primitive element, and if /14 = y, 
then H is not a proper subalgebra, H — L(x, y, z). 

By Lemma 13.4.2, u is a test element of L(x,y,z). 

7. If x and y are even and z is odd, then using the same arguments as in 
point 6 of the proof, we get that [z, y] + [[z, x],x] is APE and a test element 
of L{x,y\z). 

If y and z are even and x is odd, then [z, y\ + [[z, x],x] = [z, y] + ^[x, x\, 
and it is not an APE of L(y,z\x). To show that it is not a test element, 
one may consider the retraction homomorphism cp given by p(x) = 0 , 
V>(y) = y+%[x,x\, and (p(z) = z. 

8 . In Section 12.5.2 of Chapter 12, it was shown that u = x + [y, z] + 
(ad x) p (z) is not a primitive element of L(x, y, z). Let u belong to a proper 
subalgebra H of L(x, y, z). We have u° = (ad x) p (z), and u° is a polynomial 
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in h \, . . .• , h^- If u° is a summand of h\, then obviously u is a primitive ele- 
ment of H. Otherwise, we may suppose that h\ = h\= x, h\ — (adx) 5 (z), 
s < p. Let = h 2 + ^ 2 * Consider the following element: 

vl = u — hi — (ad/ii) p-s (/i 2 ) = [y, z] - (ad/ii) p ~ s (/i 2 ). 

Then to write u as an element of H in free generators hi, we consider u' , 
u" , and so on. At some step of this expression, we get either that [y, z] is 
a summand (up to a nonzero coefficient) of some h° (and then u has the 
form agi + f(gj | j ^ i), 0 ^ a G K, and u is a primitive element of H ), or 
hj = y, hj + 1 = z for some j , and H is not a proper subalgebra of L(x, y, z). 

Now we consider the retraction homomorphism given by <p(x) = u 
and (p(y) = (p(z) = 0. Then <p(u) = u, and </? is not an automorphism of 
L(x, y, z). Hence u is not a test element. 

It remains to show that u is not an APE of L v (x,y,z). Indeed, u is 
a primitive element of L p (x , y, z), and u = x + [y + x p , z]. It completes the 
proof of the proposition. ■ 

Lemma 13.4.4 ([265]). Let F(X) be a free product of two of its proper 
subalgebras, F = A* B. Let also a be an APE of A and b an APE of B. 
Then a -\-b is not a primitive element of F. 

Proof. Let Y = {yi,...,y s }, Z = {zi,...,z t }, s+t = n = |X|, A = F(Y), 
B = F(Z), X = Y U Z. Suppose to the contrary that a + b is a primitive 
element of F(X). Since a and b are APE of A and B, respectively, /y (a) > 1 
and lz{b) > 1. 

Let G = {gi , . . . ,g n } be a free generating set of F(X), g\ = a + b. Let 
g'i, 1 < i < be the component of gi that does not depend on elements 
from Z. It is clear that g[ = a. Using elementary automorphisms, we may 
assume that nonzero elements of the set {g ' 2 , . . . , g' n } form a reduced subset 
of A, Wig'i) < W{gi) — Iy{o)) 2 < i < n. Since G is a free generating 
set of F(X), by a finite number of elementary automorphisms, we may 
transform the set G' = {g [, . . . , g f n } to the free generating set Y of A (join 
with some zero elements). Let (p be the composition of these elementary 
automorphisms. If <p(g[) 7 ^ 0, then a is a primitive element of A, which 
gives us a contradiction with the fact that a is an APE of A. 

If (p(g[) = 0, then each monomial of <p(gi) depends on elements of Z. 
We may assume that <p{g 2 )' = yiv> t(9s+i)' — Vs • Hence, using addi- 
tional elementary automorphisms, we may suppose that p(g s + 2 )' = • • • = 
p(g n ) f = 0. It is clear that the elements <p(yi), p(g s + 2 )? • • • , p{gn) generate 
the algebra B modulo the ideal of F generated by the set Y. By p(gi) ff 
we denote the component of (p(gi) that does not depend on elements of Y. 
Then the set 






generates the algebra B. But this set consists of t elements. Since the rank 
of the algebra B is equal to t and (by Theorem 11.1.1) the algebra B is 
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Hopfian, if is a set of free generators of B. Thus (p(gi )" is a primitive 
element of B. Applying the invertible automorphism <£ _1 , we have b = 
( P~ 1 ( ( p(9i )”)• Thus b is a primitive element of B. But b is an APE of B, 
and we have come to a contradiction. Thus a + b is not a primitive element 
of F(X). U 

Lemma 13.4.5 ([265]). Let F(X) be a free product of two of its proper 
subalgebras A and B, F = A* B. Let also a be an APE of A , b an APE 
of B, and H a proper subalgebra of F such that a, 6 G H. Then a + b is 
a primitive element of H. 

Proof. Let A = F(Y), B = F(Z), Y = {yi,...,y s }, Z = {zi,...,z t }. 
We may suppose that H is a finitely generated subalgebra. Using Kurosh’s 
method for construction of free generating sets of free subalgebras (see 
Section 11.1 of Chapter 11 ), we see that H = Hi * if 2 * H 3 , where Hi is 
the subalgebra of all elements of H that do not depend on Z, H 2 is the 
subalgebra of all elements of H that do not depend on y, and if 3 is some 
finitely generated subalgebra (possibly H = Hi * if 2 ). Hence a G Hi and 
b G if 2 , and Hi and H 2 are subalgebras of A and B, respectively. Since H 
is a proper subalgebra of F, Hi ^ A or H 2 ^ B. Without loss of generality, 
we may suppose that Hi is a proper subalgebra of A. Since a is an APE 
of F and a G Hi, a is a primitive element of Hi. Hence a 4- b is a primitive 
element of F. ■ 

Consider a generalized degree function p : X — > N, where N is the set of 
positive integers. Let r(X) be the free groupoid of nonassociative mono- 
mials in the alphabet X, S(X) the free semigroup of associative words 
in X, and T(X) — » S(X) the bracket removing homomorphism. We 
set p(xi...x n ) = Y^izrziPipi) f° r %u...,x n G X. If p(x) = 1 for all 
x G X, then p is just the usual length, p = £ = £x- If a G F(X), 
a = J2 a i a ii 0 7 ^ oii e K, ai are basic monomials, and aj ^ a s with 
j 7 ^ 5 , then we set p{a) = ma Xi{p(a,i)}. By a we denote the leading part 
of a: a = Y^j ^{ a )=^{a) a j a P se P ara f e cases where we use the length 
function, we denote the leading part of an element a with respect to the 
usual degree function by a 0 . 

Proposition 13.4.6 ([265]). Let F(X) be a free product of two of its 
proper subalgebras A and B, F = A* B. Let also a and b be homogeneous 
APEs of A and B relative to some generalized degree functions pi and P 2 
of A and B , respectively (pi and P 2 could be different). Then a + b is an 
APE of F. 

Proof. By Lemma 13.4.4, a + b is not a primitive element of F(X). Let 
if be a proper finitely generated subalgebra of F, and let a + 6 G if. If 
aGff, then b G if , and by Lemma 13.4.5, a + b is an APE in F. If a £ if , 
then we consider the generalized degree function p where restrictions on A 
and B are equal to pi and / 12 , respectively. Let G be a reduced set of free 




13.4. Almost Primitive and Test Elements 261 



generators of H (relative to p). Without loss of generality, we may assume 
that the leading /i-component of a + b is equal to a. By H ' we denote the 
subalgebra of F(X) generated by H and the element a. Then H f ^ F(X ) 
(by Kurosh’s method, we may construct such a set V of free generators 
of H' that a £ V ). By Lemma 13.4.5, a + 6 is a primitive element of H f . 
Therefore, by Proposition 11.4.3, a + b is a primitive element of H. Thus 
a + b is an APE of F(X). B 

Note that if a free group F of finite rank is a product of two of its proper 
subgroups A and B, F = A * B, and a and b are APEs of A and B, 
respectively, then ab is an APE of F\ see [60, 131, 132, 319]. 

Theorem 13.4.7 ([265]). Let K be a field with char AT ^ 2. 

1 . If n — 2m, then the element 

X\X 2 + X3X4 H b X 2 m-lX2m 

is an APE of F(X). 

2. If n — 2m + 1, then the element 

X\X 2 “b X3X4 "b * * • ~b X 2rn —iX 2m -b ^2m+1^2m+l 

is an APE of the free nonassociative algebra F(X); the element 

[xi,x 2 ] + [x 2 ,Xi] H 1- [x 2m -l ,X2m] + X 2m+1 

is an APE of the free Lie p-algebra L P (X). 

3. The element x\-\ b x p is an APE of the free Lie p-algebra L P (X) . 

4 . Ifn = 2m + 1 and X = {x \, . . . , x 2m +i}, then the element 

[[x 2 , £ 1 ], Xx] + [x 2 , £ 3 ] + • • • + [x 2 m, X 2m +i] 
is an APE element of the free Lie algebra L(X). 

5. Let X = {£i,...,£ n } be even variables and Y = {yi , . . . , y m } odd 
variables. 

(a) If n = 2k and m — 2l, then the element 

[xi,x 2 } + — b [x 2 k -\,x 2k \ + [yi , 2/2] H 1- [y2i-i,V2i] 

is an APE of the free Lie superalgebra L(X;Y) and of the free 

Lie p- superalgebra L P (X\ Y). 

(b) If n = 2k + 1 and m = 21, then the element 

[[x 2 , Xi ] , Xl\ + [x 2 , Z3] + • • • + [x 2k , x 2 k +l] + [2/1 , 2/2] + • • • + [j/2i-l , V 2 l\ 

is an APE of the free Lie superalgebra L{X\Y). 

(c) If n = 2k and m = 21 -b l, then the element 

[y2l+l,X 2 k-l] + [[^ 2 /+l 5 # 2 A;],£ 2 /c] + [£l , £2] + • • • 

+ [x2k-^x 2k - 2 \ + [yi, y 2 ) + • • • + [y2i-i,y2i\ 
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is an APE of the free Lie superalgebra L(X]Y). 

(d) x\ H 1- *£ + [yi , Vi] H 1- [ y m . y m ] is an APE of the free Lie 

p-superalgebra L P (X;Y). 

(e) Ifn = 2k + 1 and m = 21, then the element 

[xi,x 2 ] H h [X2k-li%2k\ + x 2 k+l + [yi » 3 / 2 ] H 1- [2/2Z-l,2/2z] 

xs an APE of the free Lie p-superalgebra L P (X\ Y). 

(f) Ifn = 2k and m — 21 -f 1, then the element 

[ x l, x 2 \~\ l"[^2fc-l j x 2 k\-\~[yi, 2/2] d K2/2Z-1, 2/2z] + [2/2Z+l , 2/2Z+l] 

is an APE of the free Lie superalgebra L(X\Y) and of the free 
Lie p-superalgebra ^(X^Y). 

(g) If n = 2k -f 1 and m = 21 + 1, then the element 

[xi,x 2 ] + [x 3 , X 4 \ + • • • + [x 2 fc_i, X 2 fc] + a^ fc+1 

+ [2/i,2/2] H 1" [ 2 / 2 Z— 1 , 2/2z] + [2/2Z+i,2/2Z+i] 

an APE of the free Lie p-superalgebra L P (X\ Y). 

All elements pointed out above are test elements of the corresponding free 
algebras. 

Proof. The elements xx , XiXj , x p are homogeneous relative to the usual 
degree, and the element [x,y\ + [[x, z],z] is /x-homogeneous for /1 given by 
(i{x) = pi{y) = 2 and p(z) = 1. Applying Propositions 13.4.3 and 13.4.6, 
we get that described elements are APEs of F(X). By Lemma 13.4.2, these 
elements are test elements of F(X). ■ 

Problem 13.4.8. Find an algorithm to recognize the APE of free algebras 
of Schreier varieties. 

Test elements in free associative algebras are considered in our Sec- 
tion 10.2. 



13.5 A-Primitive Elements of Free Lie Algebras 

In this section, we obtain a characterization of A-primitive elements of free 
Lie algebras. We prove that in free Lie algebras of odd rank there are no 
such elements, and in free Lie algebras of even rank all these elements are 
automorphic images of one particular element. 

By A we denote the left ideal of the free associative algebra K(X) gener- 
ated by the set X. Following [356], we say that an element u of the free Lie 
algebra L(X) is called a A-primitive element if the elements -u , . . . , -u 
generate A as the left ideal of K(X). 

Theorem 13.5.1 ([265]). Let K be a field, char K ^ 2, and X = 
{xi, . . . ,x n }. Let also L(X) be the free Lie algebra on X over K. Then: 
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1 . If n — 2 m, then any A - primitive element is an automorphic image 

of 

[x 1 ,x 2 ] + [*3,2:4] H h [*2m-l,*2m]- 

2 . There are no A - primitive elements of L(X) if n is odd. 

3 . The automorphism group of the algebra L(X) acts transitively on the 
set of all X-primitive elements. 



Proof. Let X = {x±, . . . ,x n }. Recall that an additive function to: Z™ — > 
Q+ is a functional if 00 (r) 7^ 0 for all r 7^ 0 (here Z+ is the set of 
all nonnegative integers, and Q+ is the set of all nonnegative rational 
numbers) . 

It is clear that an automorphic image of a A-primitive element is again 
a A-primitive element. Let 00 i — 00 (xi), 1 < i < n. We consider a partial 
ordering on rows Vt = (uo \ . . . , oo n ): we say that Q' > if oo[ > Ui, 1 < i < n, 
and Uj > uo j for some j. Now let a; be a functional with the maximal row 
such that the leading component v with respect to 00 of some automorphic 
image v of the element u has the degree 1 (now we generalize the degree 
function using the degree ooi of free generators Xi). We may suppose that 
u = v. 

It is clear that the element u has no summands of the length one (that is, 
the linear combinations of elements of X). Since u is a A-primitive element, 
it has the maximal rank and depends on x \, . . . , x n , and there are elements 
mij G K{X), 1 <i,j < n, such that 



du 

™*i— + 

ox 1 



+ m 



du 

in 

OX n 



l,...,n. 



( 13 . 5 ) 



Suppose that the length (the usual degree) of the element u is greater 
than 2 . Then it follows from ( 13 . 5 ) that for the leading component u of the 
element u (which has u;-degree equal to 1 ) there is a relation 



m 1 



du 
dx 1 



H b m n 



du 

dx n 



= 0 , 



with not all elements m i of K(X), 1 < i < n equal to zero. Then there 
is an ^-homogeneous automorphism (p of L(X) (that is, oo(ip(xi)) = oo(xi) 
for all i) such that the leading component v° of the element v = <p(xi) 
depends on less than n free generators from X. We may assume that v 
does not depend on x\. Since v depends on x\, we may consider the fol- 
lowing functional oo f . We increase the rational number uq = oo(x 1) until 
the moment when the leading component of v relative to the new func- 
tional depends on xi. We fix this value for ou[, and we set oo[ = 00 i for 
i = 2 , . . . , n. Then the leading component of the element v relative to u/ 
has the a/-weight equal to 1 . At the same time, Q' > by the first com- 
ponent, and we have come to a contradiction with the maximal property 
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of the functional uj. Thus the length of the element u is equal to 2, and 

XL — ^ ^ij \p^i ? ^ ij G 

l<i<j<n 

with not all zero coefficients Since u has the maximal rank, applying 
a linear automorphism in new free generators Y = {yi, . . . , y n }, we have 

u = [yi, y 2 ] + Y hi >yA> 6 K - 

3<i<j<n 

Bearing in mind that u is an element of the maximal rank and continuing 
this process, we get that in the case where n = 2m the initial element u is 
an automorphic image of 

[xi,x 2 ] + [x 3 ,X 4 ] H h [x 2 m -l,x 2 m], 

which is statement 1 of the theorem. And it is clear now that there are 
no A-primitive elements of L(X) if n is odd (this is statement 2 of the 
theorem). It follows from statements 1 and 2 that the automorphism group 
of the algebra L(X) acts transitively on the set of all A-primitive elements 
(which is statement 3). It completes the proof. ■ 

Corollary 13.5.2. Let K be a field , char K ^ 2, and X = { x\ , . . . ,#2 m }. 
Let also L(X) be the free Lie algebra on X over K. Suppose that ip is an 
endomorphism of L(X) such that for any A-primitive element u of L(X) 
the element (f(u) is also A-primitive. Then is an automorphism of L(X). 

Proof. Let vo = [xi,X 2 ]-\ \-[x2m-u x 2 m\ and v — <p(w). Elements w and 

v are A-primitive. By Theorem 13.5.1, there is an automorphism xjj of L(X) 
such that x/j(v) = xv. Therefore xjj(ip(w)) = xv. But xv is a test element of 
L(X). It follows that xjjp is an automorphism. Thus (p is an automorphism 
of L(X). m 



13.6 Generic Elements of Free Lie Algebras 

In this section, we construct a series of generic elements of free Lie algebras 
and present an example of an almost primitive element that is not generic. 

Let il be a variety of Lie algebras over a field K defined by the set of 
identities 9J, L = L(X) the free Lie algebra with the set X of free genera- 
tors, and let 2J(L) be the verbal ideal of L defined by 9J. An element f of L 
is said to be il-generic if / G %3(L) but / ^ %J(H) for every proper subalge- 
bra H of L. Let 21 be the variety of abelian Lie algebras. An element u of 
the free Lie algebra L is 2l-generic if u G [L, L], and if u G [if, H) for some 
subalgebra H of L, then H = L. 

Let X be a finite set. Consider a generalized degree function p: X N, 
where N is the set of positive integers. By a we denote the leading part 
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of a. We denote the leading part of an element a with respect to the usual 
degree function by a°. 

As in the situation with free groups (see [82]), since free Lie algebras of 
finite rank are Hopfian, generic elements are test elements. 

For a E L(X), by ad a and Ada we denote the operators of left and 
right multiplication, respectively, (ad a) (b) = [a, b] and (b) Ad(a) = [ 6 , a] 
for b E L(X). 

Proposition 13.6.1 ([260]). Let K be a field, char If ^ 2, and X = 
{x,y,z}. Then the element u = [x,y\ + (x)(Adz) k ( k > 2) is an APE of 
the free Lie algebra L(X) over K. It is also a test element of L(X). 

Proof. Suppose that the element u belongs to a finitely generated subal- 
gebra H of L(X). Let x > y > z, and let (hi, . . . , h m } be a reduced set 
of free generators of H. Then the leading part u° of a is a polynomial in 
h° h° 

It is clear that u° = (x)(Adz) k . If in the expression of u° we have 
a nonzero linear summand h° for some i, then u° = ah° + f({hj \ j ^ i}), 
0 7 ^ a E K; therefore u = ahi + f'({hj | j ^ i}), and a is a primitive 
element of H. Otherwise z G H. 

Now we consider the generalized degree function /i given by jjl(x) = 1, 
and jji(y) = N > k, fi(z) = 1. Let {h[, , h' m } be a reduced set of free 
generators of the subalgebra H with respect to /i. We have u = [x, y), and 
a is a polynomial in /i' l5 . . . , h' m . If u has a nonzero linear summand hj for 
some j , then 

u = 0h j +g({h i | i^j}), 0 ^ (5 E K\ 

therefore u = (3hj d- g' ({hi \ i ^ j }), and a is a primitive element of H. 
Otherwise x G H. 

If x,z e H , then v! = u — (x)(Adz) fe = [x,y\ E H. If ( u f )° has a linear 
summand in the expression in /ij, . . . , h° m , then u is a primitive element 
of H. Otherwise y E H, and H = L(X). Thus u is an APE of L(X). Since 
the component of degree one of u is zero, by Lemma 13.4.2 u is a test 
element of L(X). ■ 

Proposition 13.6.2 ([260]). The element 

u = (x)(Ady) k + (x)(Adz)\ where k and l are positive integers, 

is an APE of L(x, y , z) if and only if min{fc, 1} = 1 and ma x{fc, 1} >2. 

Proof. If k = l = 1, then u = [x,y + z], and u is not a primitive element 
of the proper subalgebra of L(x, y , z) generated by x and y + z. 

Suppose that k,l > 1. Let u\ = (x)(Ady) k ~ 1 , U 2 = (^)(Adz) /_1 , us — y , 
= z. Then U = {^ 1 ,^ 2 , 1 x 3 , 14 } is a reduced set, u = [ 14 , 14 ] + [ 163 , 1 x 4 ], 
H = 1 /( 1 x 1 , 162 , 163 , 164 ) 7 ^ L(x,y,z ), and u is not a primitive element of H. 
Hence u is not an APE of L(x, y , z). Using Proposition 13.6.1, we complete 
the proof. ■ 
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Now we consider elements 

u k,i{ x i y) = (adx) fe (y) + (a) (Ad yf 

of the free Lie algebra L(x, y). If k = l = 2, then ^ 2 , 2 ( 2 ^ y) = [[#, 2/], £ — y], 
and this element is not an APE of L(x, y). If char AT = p > 2, then 

u p , p (x,y ) = + y, (ad p_1 (x + j/))(y - a:)], 

and u PiP (x, y) is not an APE of L(x,y). 

Proposition 13.6.3 ([260]). If k,l > 2 and k ^ l, then the element 
Uk,i(x,y ) = (ad x) k {y) + (x)(Ad y) 1 
is an APE of L(x,y). 

Proof. We may suppose that k > L In this case, u° kl {x,y) = (ad x) k (y). 
Suppose that the element u belongs to a finitely generated subalgebra if 
of L(X). Let (hi, . . . , h m } be a reduced set of free generators of if. Then 
the leading part u° of u is a polynomial in h \, . . . , h° m . As in the proof of 
Proposition 13.6.1, if u is not a primitive element of if , then x E if. In 
this case, we consider the generalized degree function y given by y(x) = 1, 
y(y) = N > k. Let {h [, . . . , h' m } be a reduced set of free generators of the 
subalgebra if with respect to y. We have u = (x)(Ady) 1 . Again, either u 
is a primitive element of AT, or y E if. If y E if, then if = L(x,y). It 
completes the proof. ■ 

It is clear that an element u of [L,L\ is 2l-generic if and only if u = 
where a* E K and the elements {/*,#*} generate the algebra 
L = L(X). Therefore, if u = Yli=i a i[ u ii v i]i where Vi are monomials 
of degree greater than one, Qj G if, then u is not a 2l-generic element of 
L(X). 

Proposition 13.6.4 ([260]). Let u be an APE of L — L(X), u E [L,L\. 
Then u is a 21- generic element of L. 

Proof. Assume that u E [if, if] for some proper subalgebra if of L. Since 
u is an APE of L,u is a primitive element of if. But any free generating set 
of if forms a linear basis of if modulo [if, if] . This contradiction proves 
the statement. ■ 

Theorem 13.6.5 ([260]). Let K be afield, char A ^ 2, X = {x \, . . . ,x n }, 
and L(X) the free Lie algebra over K. 

1) If n — 2m, then 

[xi,X 2 \ H 1- [X 2 m-l,x 2 m] 

is a 21- generic element of L(X). 

2) If n = 2m, hi, k, i = 1, . . . , m, are integers, hi, k >2 , ki ^ L, then 

UkiJi 0^1 } ^ 2 ) + * ' ' + 'U'kmilm (^TTl— 1 > 771 ) 
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is a 2t - generic element of L(X). 

3) If n = 2m + 1 and l > 2 , then 

[xi,x 2 ] + (ari)(Ada;3) i + [x 4 ,ar 5 ] + • • • + [x 2m ,x 2 m+i] 
is a ^-generic element of L(X). 

4) Ifn = 3 m, Zi, . . . , l m > 2, then 

[xi,x 2 ] + (*l)(Ad X 3) 11 + ... [«3m-2,*3m-l] + (X 3 m- 2 )(^dx 3m ) lm 
is a 21 - generic element of L(X). 

Proof. Since > 2, the elements Uk i ,i i (x 2 i-i,x 2 i) are /i-homogeneous, 
where the generalized degree function y is given by y(x 2 i-\) = k — 1 
and /x(#2i) = h — 1. The statements of the theorem follow from Proposi- 
tions 13.4.6, 13.6.1, 13.6.3, and 13.6.4. ■ 

Note that if n is even, then [xi,x 2 ] • [^3,^4] [x n _i,£ n ] is a 2l-generic 

element of the free group on xi, . . . , x n [100, 318, 375]; see our Section 3.2. 

Let X = {x,y}, x > y, L = L(x,y), L (1) = [L,L], L ( 2) = [L^, L (1 >]. By 
Theorem 11.2.8 of Chapter 11, the monomials 

((z)Ad*(s/))Ad J (a:), i > 1, j > 0, (13.6) 

form a set of free generators of the free Lie algebra L ^ (see also [32]). 

Lemma 13.6.6. Let £ be a nonzero linear combination of x and y, L = 
L(x,y). Then: 

1. If [g,£\ E L for a homogeneous element g of L of degree 4 , then 
9 = 0. 

2. If [[[a;, 2/], #],<£] = [f,£ i] for some homogeneous element f € L of 
degree 3 and a linear element £\ of L, then either [£,£i] = 0 or 
[£ u x\ = 0. 

Proof. 

1. We may assume that £ — x. The element g can be written in the left 
normed form 

g = ai{x, y, y, y] + a 2 [x, y, y, x] + a 3 [x, y, x, x\. 

Hence 

\g,l ] = a 1 [x,y,y,y,x\ + a 2 [x,y,y,x,x] + a 3 [x,y,x,x,x\ € L (2) . 

Since the elements (13.6) are linearly independent modulo L^ 2 \ ot\ = a 2 = 
03 = 0, and g = 0. 

2. Let 



( = aix + a 2 y, (oi,a 2 ) ^ (0,0), 

ti = Pi x + fcy, (/?i,/3 2 ) ^ (0,0), 

f = Hi[[x,y\,x) + j 2 [[x,y\,y\, (71,72) ^ (0,0). 
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Then the equality [x,y,x,(\ = [f,£ 1] is equivalent to the system of equations 

= hPi, ol 2 — 72/?i + 7i#2, I2P2 — 0. 

If P2 = 0 , then l\ = /?ix, and if 72 = 0 , then l = 7^1. ■ 

Theorem 13.6.7 ([260]). The element 

u = [x, y ] + [[x, y],x] + [[[x, y],x], [x, y}\ 

is an 2 t- generic element of the free Lie algebra L = L(x , y), but at the same 
time it is not an APE of this algebra. 

Proof. The element u belongs to the proper subalgebra of L generated 
by [x,y] and [[x,y],x]. It is clear that u is not a primitive element of this 
subalgebra. Therefore u is not an APE of L. 

Suppose that u is not an 2 l-generic element of L. Then there is a proper 
subalgebra H of L such that u G [H, H]. By H° we denote the subalgebra of 
leading terms of elements of H. Since H is generated by some reduced set, 
if / € [H,H\, then f° G [H°,H% and = [[[x, y], x], [x, y)} G [H°,H°). 
Hence u is a linear combination of homogeneous elements of degree 5 of the 
form [/,#], where /, g G H°. We may assume that /x(/) > //(#), where // is 
usual degree function. If fi(g) = 1 , then g is a linear element of L. If H° 
contains tyro linearly independent linear elements, then H — L. Therefore, 
we may suppose that u° = [/, [x,y]] + [g,(\, where £ is a linear element. 
Then \g,£\ G L^ 2 \ By Lemma 13 . 6 . 6 , \g,£] = 0 . Hence / = [[x,y],x], and 
[[x, y] , x] , [x, y] G H° , and H contain elements of the form a = [[x,y],x]+£ 1 , 
b = [x, y]+£ 2, where £\ and £2 are linear elements. Consider the element 

w = u-[a,b] = [x, y] + [[x, y\,x + £ 1] - [x, y, x, £ 2 \ G [. H , H\. 

Suppose that £2 — 0 . If x + £\ ^ 0 , then w° = [[x, y\,x + £ 1] G [H°, H°]. It 
follows that x + l\ G H. Therefore w — w° = [x, y] G [. H , if], and H = L. If 
x + £\ = 0 , then [x, y] G [#, JjT], and again H = L. 

If £2 7^ 0, then 



w 0 = -[x,y,xJ 2 ]e[H 0 ,H 0 }. 

This is possible only if [x,y,x,^2] = where £ is a nonzero linear 

element of if, and g is a homogeneous element of H° of degree 3 . By 
Lemma 13 . 6 . 6 , either £ = a£ 2 or £ = ax. 

If £2 G if, then [x, y] = b - £ 2 G if. As in the case ^2 = 0 , we get 
H = L. Let f = x G if. Then we may assume that a = [[x,y],x] + fly, 
b = [x, y] + 72/ G if, and 0^7 € K. Therefore 

a - [b, x] - 76 = (/? - 7 2 )y € 

Since H ^ L, — 7 2 , and 

w = u - [a, 6] = [x, 2/] + [[x, y], x + 7 2 y] - 7[x, y, x, y] € [tf, H\. 
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It follows that w° = -7 [x,y,x,y] G [H°,H°]. The element [x,y,x,y] has 
the unique such presentation [x,y,x,y] = [x,y,y,x]. Therefore [[#, 2 /],y] G 
H° , and c = [[x, y], y] + Sy G H. Consider the element 

Wi = w + 7 [c, x] = (1 - 7 < 5 )[x, y] + [[x, y], a + 7 2 y\ e [ H , H]. 

We have wf = [[x,y],x + J 2 y\ € [H°,H°]. Therefore x + 7 2 y € H. Since 
7 + 0 , = L. 

We showed that in all cases H = L. Thus u is a 2l-generic element of L. 



Let 9t m be the variety of nilpotent Lie algebras of degree not less 
than m. This variety is given by the identity [xi,X 2 ,-- - ,x m ] = 0, where 

\x\ , X2 5 ^ 3 ] = [[*^1 ? *£ 2 ] j *^ 3 ] v • • 5 [^1 ? ? 3? m ] [[^1 j ... 5 %m— l] ? %m\ • 

Theorem 13.6.8 ([260]). Let X = { 3 ^ 1 , . . . ,x n }. T/ie element u = 
[a?i, ... , x n ] is an yt n -generic element of the free Lie algebra L(X). 

Proof. Let H be a subalgebra of L(X) and {hi , . . . , h m } a reduced set of 
free generators of H. Consider the lower central series of H : 

72 (H) = [H,H\,..., li+ i(H) = 

Suppose that the element u belongs to j n (H). Then 

2=1 

where . . . ,g$ G if, 1 < i < fc. It follows that the element u° = u is 
a Lie polynomial of degree n in hi , . . . , h^. It is possible only if u depends 
on h ^, . . . , hi n , nih^) = • • • = /x(fti n ) = 1. But in this case H = L. Thus u 
is an 7t n -generic element. ■ 

For free groups, the statement of Theorem 13.6.8 was proved by V. Shpil- 
rain [352]. Note that [xi,x 2 ,xs] is an ^-generic element of L(x i,x 2 ,xs), 
but it is not an APE. 

Problem 13.6.9. Give complete characterization of generic elements of 
free Lie algebras. Find an algorithm to determine generic elements. 

L. P. Comerford [82] proved that for a variety of groups with algorith- 
mically decidable word problems there is an algorithm to recognize generic 
elements. 




14 

Free Leibniz Algebras 



Leibniz algebras are possible non- (anti) commutative analogs of Lie alge- 
bras. These algebras have appeared in [55] under the name “D-algebras” . 
In [221, 222, 223] J.-L. Loday and T. Pirashvili studied these analogs from 
the point of view of homological algebra. 

Let K be a field and L a X-algebra with the bracket multiplication [ , ]. 
Then L is a Leibniz algebra if 

[x,[y,z\] = [[x,y\,z\ - \[x,z\,y\ 

for all x,y,z £ L. 

It is obvious that any Lie algebra is a Leibniz algebra, and a Leibniz 
algebra is a Lie algebra if the condition ([x, x] = 0 for all x G L) is fulfilled. 

In this chapter, we consider the variety of Leibniz algebras. This vari- 
ety is not a Schreier variety of algebras. Nevertheless, many combinatorial 
properties of this variety are similar to the properties of Schreier varieties 
of algebras. 

In Section 14.2, we prove that the variety of all Leibniz algebras has the 
property of differential separability for subalgebras, Theorem 14.2.1. 

We note that Theorem 14.2.1 gives us a negative solution of Problem 2 
from [388]: the variety of all Leibniz algebras is not a Schreier variety but 
has the property of differential separability for subalgebras. In the proof of 
Theorem 14.2.1, we use some basic facts about standard bases of ideals of 
free algebras that are not included in this monograph. For these results, we 
refer to the monographs [33, 57, 275]. 

Theorem 14.2.2 shows that the Jacobian conjecture has a positive 
solution for free Leibniz algebras. 
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In Section 14.3, we study finitely generated subalgebras of the free Leib- 
niz algebra. An algebra R is called residually finite if for any element a G R 
and for any finitely generated subalgebra B such that a £ B there exist 
a finite dimensional algebra H and a homomorphism cp: R-> H such that 
<p(a) £ (p(B). Theorem 14.3.8 shows that free Leibniz algebras are residu- 
ally finite. It follows that the occurrence problem for free Leibniz algebras 
is decidable. 

In order to clarify differences between the variety of Leibniz algebras and 
Schreier varieties of algebras, we consider automorphisms of free Leibniz 
algebras in Section 14.4. 

Automorphisms of finitely generated free algebras of Schreier varieties 
of algebras are tame; that is, they are compositions of elementary auto- 
morphisms (see Section 11.1 of Chapter 11). To the best of our knowledge, 
at the moment there are no examples of a non-Schreier variety of algebras 
all free algebras of which have only tame automorphisms. Possibly, this 
property characterizes a Schreier variety of algebras. 

Automorphisms of two-generated polynomial and free associative alge- 
bras are tame (as was proved by H. W. E. Jung [164], W. van der Kulk [194], 
A. J. Czerniakiewicz [86], and L. G. Makar-Limanov [233]). Examples of 
wild automorphisms were constructed for the variety of matrix algebras of 
order 2 (G. Bergman [45] and also the recent article [208] by W. Lee), 
for the varieties of metabelian, polynilpotent Lie algebras (V. Drensky 
and C. K. Gupta [101, 105], Yu. A. Bahturin and S. Nabiyev [35], and 
Yu. A. Bahturin and V. Shpilrain [36]), and for some nilpotent variety 
of algebras. For more details, we refer to the review article [102] and the 
monograph [103] by V. Drensky. 

We study automorphisms of the free two-generated Leibniz algebra. In 
Section 14.4.1, Theorem 14.4.5 gives a characterization of tame automor- 
phisms. Section 14.4.2 collects corollaries of the main result. Theorem 14.4.7 
shows that there is an algorithm to recognize tame automorphisms. 
Theorem 14.4.8 gives an example of a wild automorphism. 



14.1 Basic Notions 

Let X be a set, F(X) the free nonassociative algebra on X over if, and I 
the two-sided ideal of F(X) generated by the elements 

[a, [6,c]] - [[a,b],c] - [[a,c],fc] 

for all a, 6, c 6 F(X). Then the algebra L(X) = F(X)/I is the free Leibniz 
algebra. 

The free Leibniz algebra also has another presentation. Let V be the 
vector space with the basis X. Then the tensor module 

T{V) = V © V' 02 © • • * © V® n 0 . . . 
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equipped with the bracket multiplication defined inductively by 

[x, v\ = x 0 v for x e T(U), v G V, 

[x, y (g) u] = [x, y] 0 v - [x <g> v, y] for x, y G T(U), u G V, 

is the free Leibniz algebra on X. 

Let l) and U be two copies of the Leibniz algebra L. We denote by 
L x and R x the elements of L l and U corresponding to the universal op- 
erators of left and right multiplication on x. Let I be the two-sided ideal 
of the associative tensor AT-algebra T(L l ® U ) with the identity element 
corresponding to the relations 

^[ x ^y\ — RxRy RyRxi (14.1) 

^[x^y] = LxRy Ry^x-) (14*2) 

(R x + L x )L y = 0 for any x, y G L. (14.3) 

Then the factor algebra UL(L) = T(L l ® L r )/I is the universal multi- 
plicative enveloping algebra of the algebra L in the variety of Leibniz 
algebras. 

Let L° be the ideal of the Leibniz algebra L generated by the ele- 
ments [x,x], where x G L. Then the factor algebra L = L/L° is a Lie 
algebra. We denote by x the image of an element x G L under the 
canonical homomorphism L — » L. Using this homomorphism, we have the 
homomorphism 

UL(L) -> U(L), 

where U(L) is the universal enveloping algebra of the Lie algebra L. In 
all of what follows we identify the Lie algebra L with the corresponding 
subalgebra of U(L). Then we get R x = x, L x = — x. 

Consider the homomorphism 

5: U(L)-+UL(L) 

given by S(x) = i? x , x G L. 

Since 6(x) = R x = x, we identify the algebra U(L) with the 
corresponding subalgebra of UL(L). 

The algebra UL(L) is a free right U (L)-module with the basis consisting 
of 1 and the elements L x , where x runs through the basis of L. 

From (14.2) and (14.3), we get 

L[ x ,y] = L x Ry + LyL x , where x, j/ G L. (14.4) 

Consider the mapping 

d:L-+UL{L) (14.5) 



given by 



d(x) = L x with x G L. 
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Bearing in mind (14.4), we obtain 

d([x, y ]) = L[ XtV ] = L x R y + L y L x = d(x)R y + d(y)L x for all x,y e L. 

Therefore, the mapping d is a derivation of the algebra L with coefficients 
in the UL(L )- module UL(L). We set 

II = {L x | x E L} • UL(L)\ 

i.e., Il is the right ideal of the algebra UL(L) generated by the elements L x , 
x E L. The mapping d: L II is the universal derivation of the algebra L 
in the variety of Leibniz algebras. If H is a subalgebra of a Leibniz algebra L, 
then we set 



Jh = {L x \ x e H} • UL(L). 

For more details concerning enveloping algebras of Leibniz algebras, we 
refer to [223]. 



14.2 Differential Separability of Subalgebras 

Theorem 14.2.1 ([258]). Let x be an element of a Leibniz algebra L and 
H a subalgebra of L. Then x E H if and only if d(x) E J# . 

Proof. We define subsets A, B, C, E of the Leibniz algebra L as follows: 

1) A is a basis of the subspace L° fl H. 

2) A U B is a basis of H. 

3) A U C is a basis of L°. 

4) AuBUCUBisa basis of L. 

Consider a linear ordering on the subset AU B U C U E such that from 
a E A, b E B, c E C, and e E E it follows that a < b < c < e. Let 
B = {b\be B}, E = {e \ e E E}. Then the subset B U E is a basis of the 
algebra L. By the Poincare-Birkhoff-Witt theorem, the words 

w = b h b i2 ... b in e jl e h . . . e js , (14.6) 

where b ^ < bi 2 < • • • < bi n , ej x < e j2 < • • • < ej s , form a basis of the 
algebra U(L). 

The words 



L x w, (14.7) 

where x € AUBUCUE and w is as in (14.6), form a basis of the subspace II- 
Hence the subspace II is a free (/(Z)-module with the basis 

{L x \ x e AUBUCUE}. 
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We see that the words from (14.6) and (14.7) form a basis of the algebra 
UL(L), and the right ideal Jh of the algebra UL(L) is generated by the 
elements 



L x , xeAuB. (14.8) 

Therefore the right U (Z)-module Jh is generated by the elements 
from (14.8) and the elements L x L y , where x G AuB and y G AUBUCUE. 
From (14.4) we get 

L x L y ~ LyR x + L\y^ x ] = S(x, 2 /), 

where 

= LyX — 

If x G A, then R x = 0, and L x L y — 0. Since the subset A U B is a basis 
of the subalgebra B, for y G AuB the element L x L y belongs to the U(L)- 
submodule generated by the elements from (14.8). Thus the elements (14.8) 
and 



S(x,y ), where x G B, y G C U B, (14.9) 

generate the U(L)- module Jh • 

The algebra U ( L ) is generated by the subset BUE and has the defining 
relations 

xz — zx — [x, z] = 0, where x,y € B U E, x > z. (14.10) 

Consider the linear ordering on the words from (14.6) and (14.7) at first 
by the length and then lexicographically. 

Elements from (14.9) and (14.10) have the unique composition (in 
the sense of the composition technique in combinatorial algebra, see 
monographs [33], [57], and [275]) of the type 

t = S(x, y)z — L y (xz — zx — [x, z]), 

where x G B, y G C U E. Since in (14.10) we set x > z, z G B, 

t = S(x, y)x + S(x, [y, z] ) + S( [x, z ] , y) - S{z, [y , x ] ) , 

the system of elements from (14.8), (14.9), and (14.10) is a stable system 
of relations (that is, all possible compositions are resolvable). Then by 
Shirshov’s Composition Lemma (see [33, 57, 349]), if / G J# , then one of 
the words 

L x , where xeAuB; L y x , where xgB, y € C U E, 

is a beginning subword of the leading word of /. Therefore, if L x G Jh, 
then the element L x is a linear combination of elements (14.8). Thus x G H. 



Theorem 14.2.2 ([258]). Let L be the free Leibniz algebra with the 
set {#i, . . . ,x n } of free generators and ^ an endomorphism of L. Then 
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is an automorphism of L if and only if the Jacobian matrix — 
i), . . . ,^(x n )) is invertible over UL(L). 

Proof. Let L be the free Leibniz algebra with the free generators 
xi, . . . , x n , and II the free C/L(L)-module with the base yi, y<i, . . . , y n . The 
universal derivation 



d: L\-¥ II 



of the algebra L is given by the rule d{xf) = yi, 1 < i < n. The partial 
derivatives ^ of an element / of the algebra L are given by the following 
formula: 



= It 6 ™®- 



i= 1 



By d(f) we denote the column ( , ^) T . Let /i,/ 2 , • • • ,/n be ele- 
ments of L. By J(/i,. . .,/ n ) we denote the Jacobian matrix (d(/i),. . .,<9(/ n )) 
of these elements. If if is a subalgebra of the algebra L, then we set 
Jh — d(H). By Theorem 14.2.1, for any x E L we have that x E H if 
and only if d(x) E Jh • Thus the Jacobian conjecture is true for the free 
Leibniz algebra. ■ 



Problem 14.2.3. Are the analogs of Theorems 12.5.5 and 12.5.8 valid for 
free Leibniz algebras? 



14.3 Residual Finiteness of Free Leibniz Algebras 

Now let L be the free Leibniz algebra with the set (xi, . . . , x n } of free 
generators. By (14.1), (14.2), and (14.3), the algebra UL(L) is generated 
by the elements 

J^X\ j J^X2 J • * * > T X n 5 Rxi 1 Ux2 > * • • » Rx n • (14.11) 

These elements satisfy the following relations: 

(R Xi + L Xi )L x . — 0, 1 <i,j <n. (14.12) 

Lemma 14.3.1 ([258]). The algebra UL(L) is defined by the genera- 
tors (14.11) and defining relations (14.12). The words 

Lx^ ^>xi 2 • • • L Xik R Xjl R Xj2 • • • Rx js > fc, s > 0, (14.13) 

form a basis of the algebra UL{L). 

Proof. We set 

■hxi ^ T X2 <c • • • < T Xn < R Xl < R X2 <c • • • <c Rx n ' 

Then the defining relations (14.12) have no compositions. Hence the algebra 
given by the generators (14.11) and the defining relations (14.12) has the 




276 14. Free Leibniz Algebras 



basis (14.13). Therefore it is sufficient to prove that the elements (14.13) 
form a basis of the algebra UL(L). 

The elements 



w = [...[[x il ,x i2 ],x i3 ],...,x ik \, k>l, (14.14) 

form a basis of the algebra L. 

Using the homomorphism S (see Section 14.1), we identify the alge- 
bra U(L) with the subalgebra of the algebra UL(L) generated by the 
elements R Xi , 1 < i < n. Then the words 

v = R Xji R Xj2 ...R Xjs , s> 0, (14.15) 

form a basis of the algebra U(L). 

The elements 



v , L w v, 

where v is a word from (14.15) and w is a word from (14.14), form a basis 
of the algebra UL(L). It follows from (14.2) that 

L w = [• • • [[L Xil > Rx i2 ] > Rx i3 ]>•••> Rx ik ] • 

Then the elements 

u , viL x .v 2 , 

where v, v 2 are words from (14.15), 1 < i < n, also form a basis of 
the algebra UL(L). By (14.12), it is possible to change all i?-operators in 
the word v\ by L-operators. Thus the words (14.13) form a basis of the 
algebra UL(L). ■ 

We set U = R Xi + L Xi1 Si = L Xi , 1 < i < n. Then the algebra UL(L) is 
generated by the elements U, and defining relations tiSj = 0, 1 < i,j < 
n. The words 



Si x Si 2 . . . Si k tj 1 . . . tj t (14.16) 

form a basis of the algebra UL(L). By S and T we denote the free associative 
subalgebras of the algebra UL(L) generated by the elements si, s 2 , . . . , s n 
and ti, ^ 2 , . . • , t n , respectively. 

Consider the following linear ordering on the words from (14.16), at first 
by the length and then lexicographically (setting si < • • • < s n < t\ < 
• • • < t n ). By / we denote the leading term of an element / G UL(L). 

Let u, v be some words from (14.16). We set u <C v if u is a beginning 
subword of the word v\ i.e., there is a word w from (14.16) such that v = uw. 
In addition, we set u ^ v if there is a word w from (14.16) such that w G T 
and v — u - w. 

Let I be the right ideal of the algebra UL(L) generated by the elements 
/i,...,/fc. Let be all elements of the form fasj, 1 < i < fc, 
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1 < j < n- Then gi € S and 

k r 

I^fiT + HgiST- ( 14 - 17 ) 

2=1 j = 1 

Consider the following system of elements: 

(14.18) 

Definition 14.3.2. A system (14.18) of elements of the algebra UL(L) 
(where gi G S', 1 < i < r) is (/, g)-uncancellable if the following conditions 
are fulfilled: 

a) If i 7 ^ j, then gi <C gj is not the case. 

b) For all i, j it is not true that gi fj- 

c) If i 7^ j, then fi -< fj is not the case. 

If gi <C g 3) then g 3 — g t w, where w G S'. Changing g 3 by the element 
gj — giw we get g 3 — giW < g 3 . If g x <C f 3 and fj = giw, then we change fj 
by fj — giW. If now fi -< f 3 and fj = few, w E T, then we change fj 
by fj — fiW. Using such reductions and omitting zero elements, we reduce 
the system (14.18) to an (/, #)-uncancellable system. The reductions above 
preserve the equality (14.17). 

In all of what follows, we suppose that the system (14.18) is (/,#)- 
uncancellable. 

Lemma 14.3.3 ([258]). If h e I, then there exists either an element fi 
such that fi -<h or an element gj such that gj <C ft. 

Corollary 14.3.4 ([258]). The occurrence problem in finitely generated 
right ideals of the algebra UL(L) is decidable. 

Using Theorem 14.3.1, we have: 

Corollary 14.3.5. The occurrence problem (in finitely generated subalge- 
bras) is decidable for free Leibniz algebras. 

Definition 14.3.6. We say that an element h of UL(L) is irreducible with 
respect to the system (14.18) if there are no such that fi ^ hoi gj h. 

Lemma 14.3.7 ([258]). If h £ I, then there exists a right ideal J of the 
algebra UL(L) such that J has finite codimension in UL(L), I C J 1 and 
h£J. 

Proof. We may suppose that the system (14.18) is (/, g)-uncancellable. 
Moreover, by Lemma 14.3.3, one can assume that the element h is irre- 
ducible with respect to the system (14.18). Let m be the maximal length 
of the elements 



hi fi , . . . 5 fk ) gi ? • • • ^ gr • 
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Consider the set of all basic words of the length m + 1 

fk+ 1 5 fk -\- 2 j • • • j fk\ 5 ^r+l ? 9r-\-2 5 • • • 5 *7ri 

such that they are irreducible with respect to the system (14.18), gi E 5, 

/,• i s. 

Then the system 

/l? /2) • • • j jffc) /fc+l} • • • 5 fki •> 927 • • • i 9ri 9r+ 1? • • • 5 *7ri (14.19) 

is (/, g)-uncancellable, and the element h is irreducible with respect to the 
system (14.19). 

By J we denote the right ideal of the algebra UL(L) generated by the 
elements (14.19). Since faST = f{F with k + 1 < i < &i, 

J = t^ T + t^T. 

i= 1 j= 1 

For any word of length more than ra, there exists either fi, 1 < i < 
fci, such that /i < w, or 1 < j < ri, such that ^ <C w. Then the 
algebra UL(L) is linearly generated by the elements of the length less than 
ra + 1 (modulo J). Hence the ideal J has finite codimension in UL(L). By 
Lemma 14.3.3, h £ J. ■ 

Theorem 14.3.8 ([258]). The free Leibniz algebras are residually finite. 

Proof. Let L be a finitely generated free Leibniz algebra. Let if be a subal- 
gebra of L generated by elements hi, . . . , hk. Suppose that h £ H. Then, by 
Theorem 14.2.1, d(h) = Lh Jh> Since d is a derivation, the right ideal Jh 
is generated by the elements d(hi), d(/i 2 ), . . . , d(h^). By Lemma 14.3.7 there 
exists a right ideal J of the algebra UL(L) such that J has finite codimen- 
sion in UL(L), d(h ) £ J, and Jh Q J • Then M = UL(L)/J is a finitely 
generated UL(L)- module. Let Ann (M) be the annihilator of the module M. 
Then 



Ann (Af) C UL(L) • Ann (Af) C J H • UL{L)j Ann(M) 
is a finite-dimensional algebra. Therefore the subspace 
N = {x € L \ R X ,L X E Ann (Af )} 

has finite codimension in L. Moreover, by (14.1) and (14.2) the subspace N 
is an ideal of the algebra L. Let G = L/N. Now, to complete the proof it is 
sufficient to show that h £ H+N. On the contrary, assume that h E H+N. 
Then 



d(h) E d(H) + d(N) C J H + Ann (Af ) C J, 

and we come to a contradiction. Let <p: L — > G be the homomorphism 
given by p(x) = x + N for x E N. We have that G is a finite-dimensional 
algebra, and p(h) £ p(H). ■ 
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14.4 Automorphisms of Free Leibniz Algebras 

In this section, we consider automorphisms of two-generated free Leibniz 
algebras. By A = LB(x i,x 2 ) we denote the free Leibniz algebra with free 
generators X\ and x 2 . The monomials 

ij = 1,2, 1 <j<k, 

form a linear basis of A. By UL(A ) we denote the universal multiplicative 
enveloping algebra of A. 

The algebra A has a structure of a UL(A)-mod\ile given by a-r b = [a, 6], 
a - l b = [6, a] for a, b G A. By Ann(A) we denote the ideal of A generated 
by elements {[a, a] | a G A}. Then r a = 0 if and only if a G Ann(A). 
The algebra L = A/ Ann(A) is the free Lie algebra in two generators. For 
simplicity, we denote these free generators by x\ and £ 2 . 

For an element / of A , by £(f) we denote the degree of /. Let / be the 
leading homogeneous component of / and / the linear component of /. For 
an automorphism <p of A given by p(x 1) = fa, p(x 2 ) = / 2 , we will use the 
notation p = (/1, / 2 ). The degree t(p) of an automorphism p = (/1, / 2 ) is 
the number £(p) = £(fa) + ^(/ 2 ). By (a \, . . . , a k) we denote the subalgebra 
of A generated by the elements ai, . . . , a& G A. 

Lemma 14.4.1 ([3]). For an automorphism p = (/i,/ 2 ) of A, we have 
fa — fa ^ Ann(A), i = 1, 2. 

Proof. Automorphisms of the free Lie algebra L are tame. Since L has 
rank 2, these automorphisms are linear. Hence the automorphism p of A 
induces a linear automorphism of L. Thus fi — fi G Ann(A), i = 1,2. ■ 

Lemma 14.4.2 ([3]). Let (/1, / 2 ) be an automorphism of A, T(x) G LB(x) 
with £(T(x)) = k. Then £(T(fa)) = i{fi) + k- 1. 

Proof. Let T(x) = where <*i G F, o^-i 7^ 0. By 

Lemma 14.4.1, fa — fa G Ann(A), and therefore r/ x = r^. Hence 

fc-l fc-l 

T (/i) = J2 a ih r h = '52 a ih r% f 1 > 

i — 0 i=0 

that is, T(A) = afc-i/ir)- 1 . Thus ^(r(A)) = + * - 1- ■ 



Simple automorphisms 

We recall that an elementary transformation of a set {/i, / 2 , . . . } is a change 
of one element fa by an element a fa + 0, where 0 / a G F j G ({/j | 
j 7^ i}). If {<71,02} is result of applying one elementary transforma- 
tion to {/i,/ 2 }, then we write (fa ,/ 2 ) -A (01,02). If (fa ,/ 2 ) is a tame 
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automorphism of A, then there is a series of elementary automorphisms 

(*i,* 2 ) = (/f 0 ) ,/f) (/i ( 1 ) ,/i 1} ) -»• • • • -> = (/i,/ 2 ). 

Definition 14.4.3. Let (/i,/2) be an automorphism of A. We say that 
the element /i is (elementary) reducible if there are a G F, a ^ 0, and 
9 € (/b) such that ^(a/i + p) < ^(/i). We say also that the element fi is 
reducible by the automorphism (a/i + g, /2). 

Definition 14.4.4. We define simple automorphisms of A by induction on 
the degree as follows. 



1. Linear automorphisms of A are simple. 

2. Suppose that we have defined simple automorphisms of degree less 
than k (k > 2). An automorphism (/i, jfe) of A of degree k is simple 
if fi or /2 is reducible by a simple automorphism. 

An element fi of the automorphism (/i, $2) is said to be simple reducible 
if it is reducible by a simple automorphism. 

Theorem 14.4.5 ([3]). Tame automorphisms of the free two-generated 
Leibniz algebra A = LB(x 1,2:2) are simple. 



Proof. Assume the contrary. Then there is a sequence of elementary 
automorphisms 



(*.,*.) = -+ (/r./n -*•••-+ ur.m -+ (/; 

< /(&+ 1) *(&+!)■* 



f (i) f (i) 



(fc) ,(fe)s 



p(fe+l) 



(14.20) 



such that the automorphism 0 = (/■} , ; ) is not simple, and at the 

same time all automorphisms 6i = (/i^,/^), 1 < i < fc, are simple. Let 
M be the set of all such sequences. We say that £( 6 &) = £(f[ k ^) + i( f 2^) 
is the control number of the sequence (14.20). Suppose that a sequence 
(14.20) has the minimal possible control number. We show that 6 is a simple 
automorphism. Let 



( 9 U 92 ) = (/1 



_ ( f ( k - 1 ) 



n(k 1 )\ 

5 / 2 ), 



(/i,/ 2 ) = (/ 1 (fe) ,/f ) ). 



We may suppose that 

h = fi k+1) , A k+1) = f = f2+g, 9 = T(fi) e (/1). (14.21) 

Since 6 k is simple, fi or /2 is simple reducible (by 0^). Assume that / 2 is 
simple reducible. Then we may suppose that /2 is reducible by 6k- 1; that 
is, 9\ = /1, 92 = f2~ 5(/i), ^(02) < ^(/2). Therefore we may change the 
two last automorphisms of the sequence (14.20) by one automorphism: 

6k- 1 = (01,02) -► (0i,02 + S(gi) +T(gi)) = (/1,/) = 6. 

Since the control number of the new sequence is less than ^(0&), the au- 
tomorphism 6 = (/1,/) is simple. Hence / 2 is not simple reducible by 6k • 
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Therefore we may assume that f\ is reducible by 0k-i, 

9i = fi-S(f 2 ), 92 = / 2 , (14.22) 

If £(g) > £(f2) in (14.21), then the element f of the automorphism 6 
is reducible by the simple automorphism Ok- Therefore we suppose that 
m < W By Lemma 14.4.2, l{g) = t{T{f x )) = t{f x ) + £{T{x)) - 1. 
Therefore, we have to consider the following cases: 

(a) e(fi) = t{h), e(T(x)) = 1; 

(b) t{h) < l(f 2 ), £{T(x)) > 1. 

It follows from (14.22) that /i = 5(/2); that is, £(f2) < 1(h)- Thus we 
have to consider only the case where £(fi) = £(h), T(x) = ax , S(x) = f3x, 
a, (3 e F, g x = fi - /5/ 2 , /i = /J/2, / = /2 + a/i- Let 7 = 1 + a/?. Then 
/ = 7/2 + a#i, 7/1 — Pf + If 7 7^ 0, then we change the last two 
transformations of (14.20) by 

0fc-i — (91,92) = (#15/2) (gi,f) (/1 5 /) = 0. 

Since £(gi)+£(g2) < -^(0/c), (#1, /) is a simple automorphism. It follows from 
^(#1) + l(f) < £(fi) + £($2) = £(0k) that the automorphism 0 is simple. 

If 7 = 0, then / = agi, fi = + f3g 2 ■ Since (g,g 2 ) is a simple au- 

tomorphism, (051,02) = (/5P2) is also a simple automorphism. Moreover, 
the automorphism (g2,f) is simple. 

Consider the elementary transformation (#2,/) (/1,/) = 0- Taking 

into account that 



^(<72) + £(f) — £(92) + l(gi) < £(@k- 1)? 



by the choice of (14.20), 0 is a simple automorphism. 



14-4-& Recognizing tame automorphisms 

Results of this section are corollaries of Theorem 14.4.5. 

Proposition 14.4.6. Let (/i,/2) be a tame automorphism of A, £(fi) + 
£(42) > 2. Then there is i, i = 1,2, and an element g of ({fj | j ^ i}) such 
that fi = g. 

Proof. By Theorem 14.4.5, the automorphism (/i,/2) is simple. Suppose 
that fi is a reducible element. Then there is g € (/b) such that £(f\ — g) < 
^(/i). Thus h=g. M 

Theorem 14.4.7 ([3]). If the field K is constructive , then there is an 
algorithm to recognize tame and wild automorphisms of A. 



Proof. By Lemma 14.4.1, the statement of Proposition 14.4.6 can be al- 
gorithmically verified. ■ 
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Proposition 14.4.6 gives us a possibility to construct a series of wild 
automorphisms of the algebra A. An example of a wild automorphism of 
the minimal degree is given by the following theorem: 

Theorem 14.4.8 ([3]). The automorphism (x\ + [[x 2 ,X 2 \,xi\,X 2 ) of A is 
wild. 

Proof. Let /i = x\ + [[x 2 ,x 2 ],xi], / 2 = x 2 . Since [[/ 2 , / 2 L /i] = 
[[x 2 ,x 2 \iXi\, (/i,/ 2 ) is an automorphism of A. At the same time, (/i,/ 2 ) 
does not satisfy the statement of Proposition 14.4.6. ■ 
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